-- Logs begin at Thu 2022-09-08 12:50:35 UTC, end at Thu 2022-09-08 14:22:01 UTC. -- Sep 08 13:36:25.293163 dv-cinder-856463-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 08 13:36:25.304756 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 8 13:36:25 2022] *** Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: nodename: dv-cinder-856463-2-iscsi Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: machine: x86_64 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: clock source: unix Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: pcre jit disabled Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: detected number of CPU cores: 4 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: current working directory: / Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: detected binary path: /usr/bin/uwsgi-core Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: your processes number limit is 47680 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: your memory page size is 4096 bytes Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: detected max file descriptor number: 2048 Sep 08 13:36:25.307576 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: lock engine: pthread robust mutexes Sep 08 13:36:25.309251 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: thunder lock: enabled Sep 08 13:36:25.309251 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 08 13:36:25.309251 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 08 13:36:25.349406 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: Python main interpreter initialized at 0x558b544639c0 Sep 08 13:36:25.349406 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: python threads support enabled Sep 08 13:36:25.349406 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: your server socket listen backlog is limited to 100 connections Sep 08 13:36:25.349406 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: your mercy for graceful operations on workers is 90 seconds Sep 08 13:36:25.349825 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: mapped 403077 bytes (393 KB) for 2 cores Sep 08 13:36:25.349902 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: *** Operational MODE: preforking *** Sep 08 13:36:25.350158 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: *** uWSGI is running in multiple interpreter mode *** Sep 08 13:36:25.350158 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: spawned uWSGI master process (pid: 88408) Sep 08 13:36:25.350384 dv-cinder-856463-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 08 13:36:25.351497 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: spawned uWSGI worker 1 (pid: 88421, cores: 1) Sep 08 13:36:25.351497 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: spawned uWSGI worker 2 (pid: 88422, cores: 1) Sep 08 13:36:25.351497 dv-cinder-856463-2-iscsi devstack@keystone.service[88408]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 08 13:36:29.856828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:29.856828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: from cryptography.utils import int_from_bytes Sep 08 13:36:29.857636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:29.857636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: from cryptography.utils import int_from_bytes Sep 08 13:36:29.858080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:29.858080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: from cryptography.utils import int_from_bytes Sep 08 13:36:29.858213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 08 13:36:29.858213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: from cryptography.utils import int_from_bytes Sep 08 13:36:29.909767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:36:29.909957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 08 13:36:29.910057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:36:29.910143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 08 13:36:29.910207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] command line args: [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:36:29.910270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] command line args: [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 08 13:36:29.910418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:36:29.910516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 08 13:36:29.910597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ================================================================================ {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:36:29.910674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ================================================================================ {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 08 13:36:29.910810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] admin_token = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.910898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] admin_token = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] config_dir = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] config_dir = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] config_source = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] config_source = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] debug = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] debug = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.911800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] insecure_debug = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.912973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_config_append = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_config_append = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_dir = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_dir = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_file = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.913953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_file = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.914959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.915956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.916958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.917070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] notification_format = cadf {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.917176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.917356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.917693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.97/identity {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.917878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.97/identity {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] publish_errors = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] publish_errors = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.918825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] standard_threads = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.919941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] standard_threads = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] strict_password_check = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] use_eventlog = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] use_journal = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.920922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] use_journal = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] use_json = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] use_json = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] use_stderr = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] use_stderr = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] use_syslog = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] use_syslog = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.921960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] watch_log_file = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 08 13:36:29.922076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922195 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.922995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.923919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.924942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.925987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.926976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.927932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.928896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.929894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.930002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.930100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.930745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.930868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.930968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.931884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.932937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.933891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.934928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.935976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.936926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.937961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938204 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.938986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.939935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.enabled = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.940958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.941890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.942949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.943996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.944973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.945975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.946602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.946719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.946834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.946950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.947989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.948940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.external = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.external = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.mapped = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.949903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.password = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.password = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] auth.token = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] auth.token = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.950974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] catalog.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.951929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.952960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.953998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.954950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955308 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.955914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.956996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.957934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.958985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.959986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.960953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.961891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.962877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.963925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.964957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.965904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.966965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.password = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.967909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.968981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.969922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.970990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.971919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.972911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.973970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.974905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.975964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.976887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.977980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] policy.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.978950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] receipt.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.979894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.980958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.981777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] revoke.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] role.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.982941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] role.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] role.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] role.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] role.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] role.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.983872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.984943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.985995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.986949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.987908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988204 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.988968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.989984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.990894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.991957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.provider = fernet {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.992682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.993993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] trust.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.994930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.995899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.connection = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.connection = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.996989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.997935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.998844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:29.999923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88422) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:36:30.000780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 08 13:36:30.000780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=88421) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 08 13:36:30.030215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.030493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.030705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.030884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.031030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.031250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.040437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.041099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.070395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.070597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.075028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.075312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.128606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.129513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.181189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.181460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.181460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.181724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.346188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.346430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.346665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.346888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.347111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.347327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.347422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.347639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.347865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.348318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.348574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.348814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.374966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.375913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.376216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.388086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.388663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.401675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.401888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.402019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.402181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.402314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.402879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88422) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.403698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=88421) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 08 13:36:30.421728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.422170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.422410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:30.422642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:30.422871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.422871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:30.423318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.423547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.423772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 08 13:36:30.424025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.424982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.433776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.434086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.434331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.434624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.436396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.436701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.436990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.437279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.441901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.445254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.445558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.445793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.445793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.446654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.446883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.447152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.447425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.447701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.447974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.448278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.448546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.449784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.450087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.450376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.450667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.450952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.451218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.451495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.451778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.452054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.452360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.452633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.472720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.474246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.474531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.476895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.478530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.478823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.486676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.486676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.486676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.487270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.487270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.487270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.487659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.487659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.490849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.491152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.493990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.494304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.494596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.494888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.495167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.495446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.495718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.495995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.497907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.498222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.498523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.498808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.499094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.499373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.499650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.499924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.512688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.512985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.516687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.517004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.517160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.520431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.520795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.521017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.524435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.524741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.525149 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.528366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.533289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.533480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.533480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.533480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.533873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.534003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.534003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.534003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.535411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.535725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.536020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.536334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.536624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.536918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.549111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.549417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.549694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.549960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.553617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.553924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.554207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.554834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.557569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.559127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.560758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.561040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.562256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.563678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.563843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.563964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.571755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.571755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.571755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.572995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.577346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.577653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.577948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.578229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.578622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.578903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.579184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.581992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.582304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.582607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.582885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.583306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.583599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.583886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.594981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.597999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.599742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.600964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.602472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.602877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.603993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.604321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.604728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.606210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.607713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.609310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.609676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.610149 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.611202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.612853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.615804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.616245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.616539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.616824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.617311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.619491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.622605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.623026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.623327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.623607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.625406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.625726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.626014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.631684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.632006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88422) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.634345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.634921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.635186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.635463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.635742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.636027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.636332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.636612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.636883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88422) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.639491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=88421) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 08 13:36:30.639736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.640035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.640352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.640643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.640922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.641203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.641485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.641765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.642041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.642327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.642666 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.642981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.643256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=88421) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 08 13:36:30.673653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558b544639c0 pid: 88422 (default app) Sep 08 13:36:30.677458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d5bd2-cbaf-4dd8-a445-1b2911f9518f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:30.677744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d5bd2-cbaf-4dd8-a445-1b2911f9518f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:30.678020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68d5bd2-cbaf-4dd8-a445-1b2911f9518f None None] PATH_INFO: `/v3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:30.679098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x558b544639c0 pid: 88421 (default app) Sep 08 13:36:30.682388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1/1] 10.222.0.97 () {50 vars in 791 bytes} [Thu Sep 8 13:36:30 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:39.214526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:39.215189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:39.215189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:39.219555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1/2] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:39 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:39.225760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:39.226632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:39.226632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:39.299220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: INFO dbcounter [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] Registered counter for database keystone Sep 08 13:36:39.322169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG oslo_db.sqlalchemy.engines [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88422) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:36:39.328992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writer thread running {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:36:39.529797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG passlib.handlers.bcrypt [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88422) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:36:39.530178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG passlib.handlers.bcrypt [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88422) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:36:39.593072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:39.594759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:39.744585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 2/3] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:39 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 520 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 08 13:36:40.241552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:40.312760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: INFO dbcounter [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] Registered counter for database keystone Sep 08 13:36:40.333140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG oslo_db.sqlalchemy.engines [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=88421) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 08 13:36:40.341364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writer thread running {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 08 13:36:40.471692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9141e36-bce2-4802-9802-2b8faaaf3709 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:40.476328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:40.476981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:40.477258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:40.477531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:40.487014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:40.540293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:36:40.540526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d253a7e6-e940-4d2d-a6a0-c0ede30134b6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:40.551237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 2/4] 10.222.0.97 () {58 vars in 1324 bytes} [Thu Sep 8 13:36:40 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:36:40.557906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:40.576725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a5e3ae-7aef-40b9-9a47-53156e718f35 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:40.581363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:40.582601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:40.582887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:40.583160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] PATH_INFO: `/v3/projects/5225debdf3f04c89923ec01541ea8b94` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:40.584903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] RBAC: Authorizing `identity:get_project(project_id=5225debdf3f04c89923ec01541ea8b94)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:40.639529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING keystone.common.rbac_enforcer.enforcer [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 08 13:36:40.639862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4071a1-57aa-4f03-b5f7-65d5641b0403 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:40.644798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 3/5] 10.222.0.97 () {56 vars in 1207 bytes} [Thu Sep 8 13:36:40 2022] GET /identity/v3/projects/5225debdf3f04c89923ec01541ea8b94 => generated 344 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:42.904023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a74a-5996-4429-aa93-aec74bc5001c None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:42.904611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a74a-5996-4429-aa93-aec74bc5001c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:42.904611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d8a74a-5996-4429-aa93-aec74bc5001c None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:42.908814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 3/6] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:42.914531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:42.914962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:42.915090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:42.935325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:42.936935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:42.945195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 4/7] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:42 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 08 13:36:43.217173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8a74a-5996-4429-aa93-aec74bc5001c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:43.229512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d8a74a-5996-4429-aa93-aec74bc5001c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:43.233794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:43.234466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.234745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.235039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.243531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:43.246009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1f1dc2-0ce1-47de-a318-876d599ce2c4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:43.255294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 4/8] 10.222.0.97 () {58 vars in 1324 bytes} [Thu Sep 8 13:36:43 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 13:36:43.262547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:43.271912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdf82d-a404-4d28-a54b-3ec88c572763 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:43.276788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:43.277425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:43.277705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:43.277976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:43.279371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:43.281265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610cb4e5-4ec1-463d-baea-cdbb29e8b4ba admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:43.282807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 5/9] 10.222.0.97 () {56 vars in 1198 bytes} [Thu Sep 8 13:36:43 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:45.606018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe7b9-ec07-46d3-b293-90feff32e955 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:45.606527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe7b9-ec07-46d3-b293-90feff32e955 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:45.606527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dabe7b9-ec07-46d3-b293-90feff32e955 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:45.608066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 5/10] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:45.616680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10088-28a9-4083-9307-666686d0cade None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:45.617448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10088-28a9-4083-9307-666686d0cade None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:45.617448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c10088-28a9-4083-9307-666686d0cade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:45.636839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-70c10088-28a9-4083-9307-666686d0cade None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:45.638125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-70c10088-28a9-4083-9307-666686d0cade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:45.644947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20ea8-0dea-44f8-9697-428a99e234fb None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:45.645230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20ea8-0dea-44f8-9697-428a99e234fb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:45.645501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a20ea8-0dea-44f8-9697-428a99e234fb None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:45.646824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 6/11] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:45.657117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:45.657628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:45.658456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:45.715105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG passlib.handlers.bcrypt [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=88421) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 08 13:36:45.715500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG passlib.handlers.bcrypt [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=88421) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 08 13:36:45.739005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 6/12] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:45.767796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:45.769468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:45.879296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 7/13] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:46.010593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c10088-28a9-4083-9307-666686d0cade None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.038900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.138025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c10088-28a9-4083-9307-666686d0cade None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.152754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.178063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.178708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.178986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.179259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.191842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:46.201718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:46.202753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:36:46.208313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 7/14] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:46.210193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.279048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e87fbe-ee67-4e13-901d-53c9c436ab76 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.281736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.324457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.328538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.329621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.329901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.330172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.345841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e27e38-190d-471d-ac03-508b76ed91db None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.395728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.396826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.396826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.396826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.397416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:46.397719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.399142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.412815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 8/15] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:46.418944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.423972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.431444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.555280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7ce41-29ee-495b-8bd5-08d7c6160e78 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.603135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 8/16] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:36:46.603979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.604707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.605026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.605312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.606563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.608064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.609831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.610456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 9/17] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:46.613340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.615742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.632877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.739690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afcea2a-c400-404b-9115-a8bc157a3b34 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.742891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47ee53-79a3-49a2-aea2-44beb2b3ffe2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.789291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.790167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.790457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.790740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.790890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.791826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.792446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.792936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.794922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.797241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.799826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 10/18] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:46.808677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.812387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.813849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:36:46.841424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:46.849827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:36:46.850931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:36:46.850931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:46.850931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:36:46.850931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 9/19] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:36:46.852987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:46.909569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.931527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:46.940293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caeb159-bbe7-4d7e-a003-6858cab03e8f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.987502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f92ac1-4fac-4876-942d-93238f61ca81 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:46.991347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:46.991888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:46.992202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:46.992471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:46.993215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.domain.id=default, user.id=d6d84dfa12274652b8eb04c682b6e60e) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:46.993521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:46.995560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c995a-7399-461e-a42a-f0bee25f6e70 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.003672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 11/20] 10.222.0.97 () {56 vars in 1322 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/role_assignments?user.id=d6d84dfa12274652b8eb04c682b6e60e&scope.domain.id=default&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 233 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:47.029283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:47.029929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:47.030212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:47.030487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:47.031182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:47.031480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:47.033001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bf3bf8-e700-4d6d-8b6c-a01c9dcfa451 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:47.054608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 10/21] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:36:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:49.568862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc951-002e-44fe-bd28-c8b71a028bdd None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.569375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc951-002e-44fe-bd28-c8b71a028bdd None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.569789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595dc951-002e-44fe-bd28-c8b71a028bdd None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.573900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 12/22] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:49.583480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:49.584968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:49.585760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:49.633905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:49.636227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:49.830604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 11/23] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:50.395078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dc951-002e-44fe-bd28-c8b71a028bdd None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.566139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595dc951-002e-44fe-bd28-c8b71a028bdd None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.650444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:50.651498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.651988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.652499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.681772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:50.693310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:50.694245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:50.694996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:36:50.696259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 13/24] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:36:50.704197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.793354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:50.882783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112bff7-27e3-47d3-8ef0-fdbc117c5305 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:50.937975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:50.938511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:50.938801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:50.939080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:50.939784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:50.940091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:50.941566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470c35db-33a1-4d6b-9b9a-4b4b10144c5f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:50.964611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 12/25] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:50.971309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:50.981824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.128770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830150-7c3e-46c1-8e16-319348b674f5 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.129055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830150-7c3e-46c1-8e16-319348b674f5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.129603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0830150-7c3e-46c1-8e16-319348b674f5 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.131235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 13/26] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:51.146396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4968-c0db-40e2-845a-d35ec134baa7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.147016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4968-c0db-40e2-845a-d35ec134baa7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.147503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f4968-c0db-40e2-845a-d35ec134baa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.187532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-918f4968-c0db-40e2-845a-d35ec134baa7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:51.189425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-918f4968-c0db-40e2-845a-d35ec134baa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:51.257827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f603a971-a856-4c0d-8be2-9bbd15d4f77b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.386130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.387484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.387999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.388541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.393308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.396375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.399703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 14/27] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:50 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:51.409966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 14/28] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:51.410201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.459547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb7ba-6ca9-4dd7-9a34-6d960f115aa2 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.460176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb7ba-6ca9-4dd7-9a34-6d960f115aa2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.460685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184fb7ba-6ca9-4dd7-9a34-6d960f115aa2 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.462357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 15/29] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:36:51.481056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad95e-e27a-46af-b4ba-6ccd97c2eb3f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.482050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad95e-e27a-46af-b4ba-6ccd97c2eb3f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.482601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bad95e-e27a-46af-b4ba-6ccd97c2eb3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.521524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-07bad95e-e27a-46af-b4ba-6ccd97c2eb3f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:51.523668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-07bad95e-e27a-46af-b4ba-6ccd97c2eb3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:51.641703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.677084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:51.791263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 16/30] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:51.794722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71819d41-3240-4ba0-87be-ee157027abcf None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:51.795102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c4f6-cfcd-45bc-a6f1-45e4aebd2eb4 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.795743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c4f6-cfcd-45bc-a6f1-45e4aebd2eb4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.796434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc00c4f6-cfcd-45bc-a6f1-45e4aebd2eb4 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.798913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 17/31] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:51.808040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.808802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.809447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.835034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:51.836575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:51.877740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:51.880017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:51.880017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:51.880017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:51.885112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:51.886852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:51.889077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 15/32] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:51.898077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:51.982084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 18/33] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:36:52.088578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce62d534-409d-4dbe-8df7-685478556ed9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.124467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.155615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.156817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.157762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.157876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] PATH_INFO: `/v3/domains/default/users/d6d84dfa12274652b8eb04c682b6e60e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.167670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d6d84dfa12274652b8eb04c682b6e60e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.170531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:52.334385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918eac5a-411a-4609-937d-93261b3f1642 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.337251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:52.403487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.404539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.405062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.405552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.436300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.439108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:36:52.475674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:52.480816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:36:52.481676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:36:52.481676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:36:52.481676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:36:52.482029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 19/34] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:36:52.483207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.589666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 16/35] 10.222.0.97 () {58 vars in 1361 bytes} [Thu Sep 8 13:36:51 2022] PUT /identity/v3/domains/default/users/d6d84dfa12274652b8eb04c682b6e60e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:36:52.591238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.674252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc34ef6f-654b-4479-99f2-360859f85d15 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.742910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63767124-92a0-4c06-b63c-200177dc9541 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:52.746402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.747507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.747998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.748452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.749579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.752042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fb43c6-e429-4c3c-b092-4f94d1caf177 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.802266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:52.803304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:52.803737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:52.804250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:52.805469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:52.806117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:52.809357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:52.848823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 17/36] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:36:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:52.855305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:52.964653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:53.042358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc2a16-c8e1-4497-a77e-8dff676cec81 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 20/37] 10.222.0.97 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:52 2022] POST /identity/v3/roles => generated 219 bytes in 598 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa01f5-70c6-46f0-b31d-39dfe025a55b None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa01f5-70c6-46f0-b31d-39dfe025a55b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afa01f5-70c6-46f0-b31d-39dfe025a55b None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 21/38] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f25fa-95e5-49b9-9510-c70b7c5c636a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.080624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f25fa-95e5-49b9-9510-c70b7c5c636a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.091511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f25fa-95e5-49b9-9510-c70b7c5c636a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.091511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 22/39] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:53.091511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f25fa-95e5-49b9-9510-c70b7c5c636a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:53.112985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:53.122470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:53.123363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.123889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.124517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.125469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:53.127132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be79746-3ad5-4e4d-ba39-a3fdb97c99a2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:53.245636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 18/40] 10.222.0.97 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:52 2022] POST /identity/v3/roles => generated 225 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:53.248397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c2b4-a4c9-4085-be8e-b968ec193e5d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.249644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c2b4-a4c9-4085-be8e-b968ec193e5d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.250940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b7c2b4-a4c9-4085-be8e-b968ec193e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.255826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f25fa-95e5-49b9-9510-c70b7c5c636a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:53.291218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-c5b7c2b4-a4c9-4085-be8e-b968ec193e5d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:53.295441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c5b7c2b4-a4c9-4085-be8e-b968ec193e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:53.375848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:53.376548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.376834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.377114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.378096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:53.380654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:53.462245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 19/41] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:53.463845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.464183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.464441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.498817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:53.501637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:53.569298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 23/42] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:36:53 2022] POST /identity/v3/projects => generated 301 bytes in 480 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:53.606456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:53.651861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 20/43] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:53.762972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:53.845520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:53.890920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c26902-7b97-4fb8-9653-6021be10b3ea None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:53.929900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:53.932894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:53.933671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:53.933962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:53.934897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:53.935436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:53.939999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:53.939999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 24/44] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:53.947466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:54.068296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba3d2da-60e5-48d1-8761-40386027e339 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:54.218198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:54.230586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:54.231596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:54.233776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:54.233776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:54.233776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:54.236130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f666572b-4797-4fd8-92d4-eb04aff37c0d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:54.281541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310d581f-a9f0-407c-9e05-4f41a410f92c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:54.342918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 25/45] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:36:53 2022] POST /identity/v3/projects => generated 312 bytes in 399 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:36:54.348758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:54.349830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:54.350330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:54.350778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:54.351989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:54.354513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb79ae0-d213-4078-9546-b215da141832 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:54.458726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 21/46] 10.222.0.97 () {60 vars in 1131 bytes} [Thu Sep 8 13:36:53 2022] POST /identity/v3/roles => generated 223 bytes in 530 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:36:55.153811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15bbc5-ec8a-46b8-8bc4-3748800475de None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.154095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15bbc5-ec8a-46b8-8bc4-3748800475de None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.154370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15bbc5-ec8a-46b8-8bc4-3748800475de None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.155801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 26/47] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:55.162149 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.162698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.162698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.182395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:55.183614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:55.227424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:55.263940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 22/48] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:55.535658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15bbc5-ec8a-46b8-8bc4-3748800475de None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.655835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15bbc5-ec8a-46b8-8bc4-3748800475de None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.695528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.696204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.696490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.696766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.710640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:36:55.717254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:36:55.718218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:36:55.719241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 27/49] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:36:55.724602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.840027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf5648-d22d-4003-a14f-89705eb96e79 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:55.880774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:55.881564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:55.881935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:55.882310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:55.883083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:55.883468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:55.885278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:55.895531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 23/50] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:55.901476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:55.951380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:56.014661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933fbfcd-2389-475a-a1ad-1d5cbc1b8ca5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:56.051794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:56.052476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:56.052827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:56.053115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:56.054559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:56.056046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:56.057758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 28/51] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:36:55 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:56.063473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:56.099359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:56.174788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc46f6-258e-4158-9c01-c9f3595c3c87 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:56.215278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:56.215934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:56.216236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:56.216516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:56.217799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:56.219285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068b6e06-838d-4529-85a0-0cf9eb9bc50a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:56.220989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 24/52] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:56.227181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:56.341877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79893700-b142-4513-95a7-dfb055bed5fe None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:56.355390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:56.380731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:56.381347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:56.381692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:56.382346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:56.382744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.domain.id=default, user.id=d6d84dfa12274652b8eb04c682b6e60e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:36:56.383049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:56.384527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94e7e53-db49-425f-8326-30830fff2d6a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:56.394733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 29/53] 10.222.0.97 () {56 vars in 1322 bytes} [Thu Sep 8 13:36:56 2022] GET /identity/v3/role_assignments?user.id=d6d84dfa12274652b8eb04c682b6e60e&scope.domain.id=default&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 530 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:36:56.929233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b886b8-4ce2-458b-820a-d369669f94d6 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:56.929519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b886b8-4ce2-458b-820a-d369669f94d6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:56.929799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b886b8-4ce2-458b-820a-d369669f94d6 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:56.931185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 25/54] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:36:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:36:56.938143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:56.938708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:56.938878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:56.970964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:36:56.973743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:36:57.100465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 30/55] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:36:57.372305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b886b8-4ce2-458b-820a-d369669f94d6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:57.491104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:57.495388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b886b8-4ce2-458b-820a-d369669f94d6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:57.534764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:57.535416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:57.535697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:57.535967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:57.537379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:57.538877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1f787-7cc0-4fc4-a87b-9e94c302883a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:57.540627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 26/56] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:36:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:36:57.547476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:36:57.666137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5dff34-bb6a-4de5-b1d7-b1800e20fcbd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:36:57.668926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:36:57.705405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:36:57.706133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:36:57.706736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:36:57.707202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:36:57.708216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:36:57.709734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431ee1ec-862f-4132-835e-2e10868922dd None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:36:57.805564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 31/57] 10.222.0.97 () {60 vars in 1140 bytes} [Thu Sep 8 13:36:57 2022] POST /identity/v3/projects => generated 298 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:00.079416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50013f4-32ff-4573-b8fe-dfa8271da5a1 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:00.079988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50013f4-32ff-4573-b8fe-dfa8271da5a1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:00.079988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50013f4-32ff-4573-b8fe-dfa8271da5a1 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:00.081549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 27/58] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:00.087753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:00.088340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:00.088340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:00.108772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:00.110145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:00.187717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 32/59] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:00.463710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50013f4-32ff-4573-b8fe-dfa8271da5a1 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:00.588606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50013f4-32ff-4573-b8fe-dfa8271da5a1 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:00.631372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:00.632024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:00.632335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:00.632613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:00.633844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:00.635332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85caa-c1fc-418f-b3f3-db3093df8f7d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:00.637068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 28/60] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:00.646023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:00.746887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:00.764189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792b997-5666-4ad2-b882-7fbc599e8895 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:00.804731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:00.805470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:00.805759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:00.806039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:00.806782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:00.808460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8526883e-bc87-4fee-baa4-4b43f605d0c0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:00.931200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 33/61] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:00 2022] POST /identity/v3/users => generated 275 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:05.074699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a240bc-fef0-4f2e-93fd-176acd180597 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.076396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a240bc-fef0-4f2e-93fd-176acd180597 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.077287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a240bc-fef0-4f2e-93fd-176acd180597 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.080347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 29/62] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:05.087478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b20-8dca-4c7e-8f79-659d8d2a3fe7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.089086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b20-8dca-4c7e-8f79-659d8d2a3fe7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.089904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb138b20-8dca-4c7e-8f79-659d8d2a3fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.133063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-bb138b20-8dca-4c7e-8f79-659d8d2a3fe7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:05.135797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bb138b20-8dca-4c7e-8f79-659d8d2a3fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:05.282179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 34/63] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:05.447926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088dc3-969f-4e56-96fa-c55c5cc10cea None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.449731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088dc3-969f-4e56-96fa-c55c5cc10cea None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.450500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42088dc3-969f-4e56-96fa-c55c5cc10cea None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.453401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 30/64] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:05 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:05.459636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921bc4-1f62-4eca-b039-b09a671710ae None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.459923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921bc4-1f62-4eca-b039-b09a671710ae None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.460480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f921bc4-1f62-4eca-b039-b09a671710ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.490028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4f921bc4-1f62-4eca-b039-b09a671710ae None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:05.491606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4f921bc4-1f62-4eca-b039-b09a671710ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:05.602800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 35/65] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:05.656344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42088dc3-969f-4e56-96fa-c55c5cc10cea None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.722041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.746556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd264-0496-4b76-a9db-d8a3e39cd68a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.747633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd264-0496-4b76-a9db-d8a3e39cd68a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.748173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3dd264-0496-4b76-a9db-d8a3e39cd68a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.749819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 36/66] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:05.773181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bdd10-0f02-4ae2-9e6b-b28c699ab5f2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.773511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bdd10-0f02-4ae2-9e6b-b28c699ab5f2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.773740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1bdd10-0f02-4ae2-9e6b-b28c699ab5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.792583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:05.809577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-aa1bdd10-0f02-4ae2-9e6b-b28c699ab5f2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:05.813414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-aa1bdd10-0f02-4ae2-9e6b-b28c699ab5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:05.828538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42088dc3-969f-4e56-96fa-c55c5cc10cea None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:05.869406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:05.870083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.870368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.871183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.884291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-dc3e90ce-abd3-4408-9d7d-2a5ee3012936 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:05.891102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:05.892090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:05.892939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:05.895130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 31/67] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:05.897267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 37/68] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:05.904306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812184-b608-4c27-85a6-fee089af1180 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.904306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812184-b608-4c27-85a6-fee089af1180 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.904306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44812184-b608-4c27-85a6-fee089af1180 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.904306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 32/69] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:05.906471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:05.907051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:05.907534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:05.925250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44812184-b608-4c27-85a6-fee089af1180 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:05.937427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:05.938701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:06.054369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 38/70] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:06.060190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.109719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.128432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44812184-b608-4c27-85a6-fee089af1180 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.150156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.199977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b67efb-2db3-42d7-8883-73f6d49ce930 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.211035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.212164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.212708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.213212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.214559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:06.215053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.217823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dc3aa7-f022-4dbd-b7b8-3fd6a3befce2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:06.232130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 33/71] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.233486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab49c4-b910-49d1-be22-ef393e6ec969 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.233798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab49c4-b910-49d1-be22-ef393e6ec969 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.234059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab49c4-b910-49d1-be22-ef393e6ec969 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.235415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 34/72] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:06.237606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab49c4-b910-49d1-be22-ef393e6ec969 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.250452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.251575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.251867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.252169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.266534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:06.272678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:06.274475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 39/73] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:06.276008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.367136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab49c4-b910-49d1-be22-ef393e6ec969 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.403994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.404677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.404961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.405238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.406070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31995556-7a7d-43c5-9e88-5357ab63a67b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.418478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-e305d041-17f2-4fb1-9d67-b2e2d95fd8aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:06.424450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:06.426039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 35/74] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:06.427556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.429016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.429016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.439456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.447465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:06.453330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:06.455115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.455859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.456322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.456829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.466030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.477312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.478996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:06.484398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 40/75] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.485592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.530649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 36/76] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:06.532854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.625558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef5318-a8d4-45b3-b7f0-9154d8638cc3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.672938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.673562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.673860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.674146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.674802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:06.675105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.675614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658690c6-bfa1-48e1-85a3-12e0522ba029 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.676538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:06.686629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 41/77] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.691515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.738348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.739379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.739773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.740271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.742411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.745094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:06.751860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 37/78] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.751860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.772058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.779802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:06.847514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9400b39-56a9-40d4-9ba0-0ee8ec8d8125 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.878027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce67d6f6-2bfe-41fa-bf6b-de73eeb469d1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:06.887132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.887773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.888051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.888432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.889339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:06.889640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.891960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:06.901382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 42/79] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:06.902838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:06.917221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:06.917887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:06.918172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:06.918451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:06.996681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:06.998178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.018283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.020881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e11a93-4853-49bb-b6f2-b78d745917fa None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.023950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.069699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.070347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.070633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.070921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.072338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.074620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.076364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 43/80] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:07.083779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.131517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 38/81] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:07.132343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.205416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd1229-32e6-4413-ac08-4552fec7ca0e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.247538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.248197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.248487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.248795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.249473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.250976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.255923 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbde2e9e-ca52-4080-b541-a4f63c0d1f72 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.276253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.296227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.296876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.297159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.297630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.310607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:07.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:07.317614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 39/82] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:07.319121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.397275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 44/83] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:07 2022] POST /identity/v3/users => generated 289 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:07.398221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.460729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.495615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da932478-a69d-4f7d-be86-fc1562f015a6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.550404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.551059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.551348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.551629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.553066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.554775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.564188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 40/84] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:07.564188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.613921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a3a302-cffe-47ea-b998-01dfc4436d4a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.620515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.667767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.668387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.668685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.669092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.732890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570a4587-6fb4-442a-aa89-c2497108954e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.775090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:07.775748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:07.776031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:07.776348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] PATH_INFO: `/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:07.780261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] RBAC: Authorizing `identity:get_project(project_id=2f12ff72a2c14db7a56eaeb9c6fd69a7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.780261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.783188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 41/85] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7 => generated 312 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:07.783188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:07.815939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:07.817525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:07.874097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:07.963488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d40e08f-7981-4d9c-a113-b29394d63b7f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:07.998740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 45/86] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:07.999608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.006230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.064419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.065949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.066228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.066501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.067203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:08.067508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.068999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.079890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 42/87] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:08.087330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.208343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e273d4-38d1-466e-a854-feafee4cc3bb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.273753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.290725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.291216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1edd281-7130-4bb3-8cac-1d228d9c1818 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.306028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.306028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.306028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.380663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.382109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.382922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.383694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.385358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=2f12ff72a2c14db7a56eaeb9c6fd69a7, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:08.386206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.390806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.409693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 43/88] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=2f12ff72a2c14db7a56eaeb9c6fd69a7&role.id=2061abea01db4efb90f840ed1ff9465d => generated 259 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:08.409693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.503940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.508416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.528874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:08.776248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daec8ced-2fa5-4037-ab4b-b3311621a261 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:08.889330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 46/89] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 890 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:08.893612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.900603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:08.901649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:08.902186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:08.902729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:08.905003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:08.907689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:08.910920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 44/90] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 505 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:08.916833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:08.997228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.178465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff1275-24dc-49cd-86dd-2ce8cd5fab71 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.204817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.211947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d3ab3c-4fe1-40d4-95b1-9040e5a476de None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.282950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.283605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.283905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.284174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.285416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.286909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.297043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 45/91] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.298001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.312849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.314440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.315240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.316055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.322768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.326403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.330796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 47/92] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:08 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 441 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.332713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.549256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:09.560389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3db5fb5-0886-4ef5-9ff1-416e20ea61e1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.651901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.653161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.653450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.653727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.654492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:09.654805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.656583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.665401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 48/93] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 259 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.666294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:09.690669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b8cfb0-7383-41ce-b34d-0e1673023988 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.776984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.778251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.780444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.780775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.781494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:09.782000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:09.783392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:09.805284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 46/94] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=2061abea01db4efb90f840ed1ff9465d => generated 259 bytes in 508 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:09.894729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6010610-94c4-450e-a8c5-ccee5701b6bf None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:09.947610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:09.948288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:09.948579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:09.949717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:09.999108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.058001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.059933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.211260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 49/95] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:10.220105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.276367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.379869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd06a06-30fd-4f66-9f59-13190d31eaa5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.438483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.439150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.439431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.439704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.441147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.442721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33286233-131b-4f4c-a308-c0f3b7d55f3b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.444551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 47/96] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:10.455095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:10.538446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:10.594245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e07593-7d4d-42ef-ba0c-17479f95ccec None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:10.642738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:10.643371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.643646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.643927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.644725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=d6d84dfa12274652b8eb04c682b6e60e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:10.645040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:10.646527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa671b5-19a5-43ae-aa27-f18ee80657e3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:10.658313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 50/97] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:10 2022] GET /identity/v3/role_assignments?user.id=d6d84dfa12274652b8eb04c682b6e60e&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:10.959495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d445-f881-48eb-a683-60cd6e567f0c None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.959934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d445-f881-48eb-a683-60cd6e567f0c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.960234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9854d445-f881-48eb-a683-60cd6e567f0c None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.961432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 48/98] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:10.972298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8eeff-818f-478c-8d72-e1cac8f8a965 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:10.972705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8eeff-818f-478c-8d72-e1cac8f8a965 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:10.972961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8eeff-818f-478c-8d72-e1cac8f8a965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:10.998424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-09c8eeff-818f-478c-8d72-e1cac8f8a965 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:10.999662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-09c8eeff-818f-478c-8d72-e1cac8f8a965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:11.105084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 51/99] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:11.591279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94874-c8de-4e3c-b8e9-973c880f6047 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.592297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94874-c8de-4e3c-b8e9-973c880f6047 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.592297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c94874-c8de-4e3c-b8e9-973c880f6047 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.593578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 49/100] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:11.601876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:11.602719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:11.602719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:11.626019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:11.627255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:11.701909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:11.722497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 52/101] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:11.878660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c94874-c8de-4e3c-b8e9-973c880f6047 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.065934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c94874-c8de-4e3c-b8e9-973c880f6047 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.080568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:12.122115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.122954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.123597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.123885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.125279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.126807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.128555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 50/102] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:12.134755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.304131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c14a3e-ea59-4244-aae7-171c1ef655d1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.354974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:12.355645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.355927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.358152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.358869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:12.361492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac06857-f769-46ba-9cb3-f129d21c451b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:12.500844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 53/103] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:37:12 2022] POST /identity/v3/projects => generated 302 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:12.605970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:12.843885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a492-9fc9-4fa1-af02-b2fc17ac335e None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.844636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a492-9fc9-4fa1-af02-b2fc17ac335e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.844763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e2a492-9fc9-4fa1-af02-b2fc17ac335e None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.847329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 54/104] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:12.854419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:12.854916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:12.855427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:12.891669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c9eb34-bed6-4640-87f4-60c887c777ad None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:12.911468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:12.913671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:13.043531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.044559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.045049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.045547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.054094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:13.093985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:13.103148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:13.104984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:13.106575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:13.113844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 51/105] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 508 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:13.122515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.169220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 55/106] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:13.200321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:13.288842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962d5da-46e0-46ea-9fea-87811eda51b1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:13.350517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.351054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.351338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.351615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.352371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:13.352678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:13.354150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:13.369941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 52/107] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:13.379132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.601019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f832be6e-ef70-44bb-917f-07a86a2ea8fb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:13.673721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:13.674821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:13.675300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:13.675772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] PATH_INFO: `/v3/users/69904edd2b3d480d904195e97f267ebd` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:13.702860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] RBAC: Authorizing `identity:get_user(user_id=69904edd2b3d480d904195e97f267ebd)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:13.705501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23333956-06ab-47f0-8da2-20a2b35a4b78 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:13.708502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 56/108] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:13 2022] GET /identity/v3/users/69904edd2b3d480d904195e97f267ebd => generated 289 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:13.717899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:13.956113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:13.962575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dc3dfc-4e10-4b51-bd94-8113aef38075 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.021515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.022154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.022442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.022725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.133137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.134675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.199533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12269-9c16-438d-8586-49ad6a49a4d4 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.200700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12269-9c16-438d-8586-49ad6a49a4d4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.201217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b12269-9c16-438d-8586-49ad6a49a4d4 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.213032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 57/109] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:14 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:14.222121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.222625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.223121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.286020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:14.288744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:14.310349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.325894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.329648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 53/110] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:14.330708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.457753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 58/111] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:14.458660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.518730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a705fdd-3455-4162-8de5-869b60964f65 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.559401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.560006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.560356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.560639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.573320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-3ea62d2f-6128-4d82-ab36-266399252c36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:14.578835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:14.580564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 54/112] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:14.582013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cca37-bc2a-4da7-829a-9bda39ecbe45 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.583442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cca37-bc2a-4da7-829a-9bda39ecbe45 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.583442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694cca37-bc2a-4da7-829a-9bda39ecbe45 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.583442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 55/113] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:14.589247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cca37-bc2a-4da7-829a-9bda39ecbe45 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.682711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.738177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694cca37-bc2a-4da7-829a-9bda39ecbe45 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.779868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.780459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.780746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.781028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.781722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:14.782029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.783479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf461df-9eff-481c-8005-ad438f065f85 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.794196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 56/114] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:14.795518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.795835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.796075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.816006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:14.817381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:14.826616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a0a32-8d5a-4806-86ff-89c118c186a4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:14.898477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 57/115] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:14.899362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.931100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:14.931760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:14.932046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:14.932381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:14.935116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:14.936818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:14.938953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 59/116] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:14.939639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:14.977292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:14.991530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.069900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379395-d641-416c-be27-5a6530d168e9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.093475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d13824-0b9c-479a-a515-5f0eb6fe4b1a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.112437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.113091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.113405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.113695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.127310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c7b922c8-e4d7-4344-9076-06b2852d1114 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:15.132693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.134469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:15.135967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 60/117] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.144183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215414b9-0ce0-46f0-b98d-cf2b8fc4507f None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.144183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215414b9-0ce0-46f0-b98d-cf2b8fc4507f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.144183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215414b9-0ce0-46f0-b98d-cf2b8fc4507f None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.144996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 61/118] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:15 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:15.145235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215414b9-0ce0-46f0-b98d-cf2b8fc4507f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.176965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.177932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.178508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.178943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.181282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.184720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.195223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 58/119] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:14 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.197016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.247970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.271727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215414b9-0ce0-46f0-b98d-cf2b8fc4507f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.314935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.315588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.315873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.316654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.317549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=69904edd2b3d480d904195e97f267ebd) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.317977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.320495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af002fd4-0317-457c-a971-717b14618e0f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.327509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 62/120] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/role_assignments?user.id=69904edd2b3d480d904195e97f267ebd&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=7b1f222929844a9b84e1979770f32290 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.329259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.329668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.330053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.351453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:15.353136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:15.443041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53769f5-b88f-4f36-a321-f9565c7e5056 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.485749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.537777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.539054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.539569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.540067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.541406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:15.541810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 63/121] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:15.541971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.548190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.548453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.566806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 59/122] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.571222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.778154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:15.812468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf1233-00b1-47e1-a34c-587c52afad12 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.838511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abafbc0-5820-4c8b-b1fc-b0cd6d6ef9b1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:15.853867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.854517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.854802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.855081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] PATH_INFO: `/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.857016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] RBAC: Authorizing `identity:get_project(project_id=2f12ff72a2c14db7a56eaeb9c6fd69a7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:15.858548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:15.860900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 64/123] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7 => generated 312 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:15.861446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.907785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:15.908486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.908789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.909033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.922312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-3b1acdee-ad9f-4752-b489-4f08e7e1b9ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:15.927934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:15.936087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 60/124] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:15.937913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d7a6f-f057-44d7-b07c-51ba84a4f3f0 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:15.938594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d7a6f-f057-44d7-b07c-51ba84a4f3f0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:15.938594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80d7a6f-f057-44d7-b07c-51ba84a4f3f0 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:15.938594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 61/125] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:15.938594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d7a6f-f057-44d7-b07c-51ba84a4f3f0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:15.989665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.018033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5d60dd-69f8-4986-a7e1-03bfeff5382b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.057797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.058623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.058783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.059574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.060926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.062463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.071473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 65/126] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:15 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.071473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.089662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.093664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80d7a6f-f057-44d7-b07c-51ba84a4f3f0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.135723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.136425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.136708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.137004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] PATH_INFO: `/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7/users/b9aec71002b149c6b1beb79cdf0db364/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.139185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] RBAC: Authorizing `identity:create_grant(project_id=2f12ff72a2c14db7a56eaeb9c6fd69a7, user_id=b9aec71002b149c6b1beb79cdf0db364, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.140701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4797ef20-cd5c-4ac8-bfd5-8363a040ba8a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.196825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6751d9c-05af-4eff-b6d5-b71093465e85 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.236711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.237365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.237698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.237976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.251332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:16.256752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:16.259052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:16.260377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 66/127] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:16.263760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 62/128] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:15 2022] PUT /identity/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7/users/b9aec71002b149c6b1beb79cdf0db364/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:16.264378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.266015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.266625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.267109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.302596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:16.305744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:16.393383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.413819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 63/129] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:16.421565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.464182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0236ed51-08b0-4e36-a499-5a833148f0e3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.491241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.557721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.558808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.559829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.560357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.563080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.563618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.569507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.586967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 67/130] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.591554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.718844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527b5317-992b-4afa-b9d7-5d70b8aa25fe None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.768746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbfc573-6de9-46bb-8273-4af7dd3253be None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:16.814766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.815940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.816836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.817407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.819455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.822564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.826189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 64/131] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.832874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:16.836085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:16.837117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:16.837569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:16.838015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:16.839145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:16.839987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:16.843757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:16.846329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:16.862530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 68/132] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:16.864297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.026816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.038651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7e0bd3-8c27-47af-90c1-554aac93bc45 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.061680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61582d5-7032-4dae-b87e-a5ef7e09ec5d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.113704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.114912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.115466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.115996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.118066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.120827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.128957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 65/133] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.130592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.131576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.132710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.133183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.133637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.135580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.138782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.142800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 69/134] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.144241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.292605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.331701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf187dd-6e72-4079-9196-5216105ee20e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.377990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5904bb54-8d7c-40cb-a886-9bf79c74470d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.398485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.399538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.400048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.400580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/users/b9aec71002b149c6b1beb79cdf0db364/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.404613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, user_id=b9aec71002b149c6b1beb79cdf0db364, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.407206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.441068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.441785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.442153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.442489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.443884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.445510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.447319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 70/135] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.448358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.616662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6008df89-2da4-4973-b081-1524b6a9e919 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.641404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 66/136] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:17 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/users/b9aec71002b149c6b1beb79cdf0db364/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 512 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:17.648702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.664634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.676762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.677860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.678342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.678814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.680947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.685237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.690971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 71/137] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.692317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.728704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:17.882044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e817fc-d847-4d42-9462-999ad1f95bca None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:17.930466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:17.931509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:17.931995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:17.932829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:17.935740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:17.937938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:17.940232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 72/138] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:17 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:17.941521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:17.962457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655c99fd-3e2f-41c4-9df9-0532f869ab6c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.037623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.056398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.057494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.057980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.058528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.060365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.063430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.082493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d81a4ed-a094-4344-b965-bfbb36bec3a4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.140832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.141928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.142468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.142958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/users/d6d84dfa12274652b8eb04c682b6e60e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.146432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, user_id=d6d84dfa12274652b8eb04c682b6e60e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.149252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d531ab-4d79-4ae7-9716-bdfb78bf07ea None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.237433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 67/139] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:17 2022] POST /identity/v3/users => generated 283 bytes in 595 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:18.239307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:18.332117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 73/140] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:17 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/users/d6d84dfa12274652b8eb04c682b6e60e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:18.440769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:18.483159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937642da-01d5-4f68-ba53-7e3cbe7c2fa4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:18.544737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:18.545888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.546382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.546863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/users/b9aec71002b149c6b1beb79cdf0db364/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.548225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed4b4-eb10-4df1-85e1-1c374f9ff6a1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.550120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed4b4-eb10-4df1-85e1-1c374f9ff6a1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.550120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eed4b4-eb10-4df1-85e1-1c374f9ff6a1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.551887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 74/141] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:18.552331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, user_id=b9aec71002b149c6b1beb79cdf0db364, role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:18.554076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37368d72-2aac-4bbc-a501-18156d3a18b8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:18.561583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8c68e-15b7-4ad8-8283-ced5d3df9d54 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:18.562037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8c68e-15b7-4ad8-8283-ced5d3df9d54 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:18.562570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf8c68e-15b7-4ad8-8283-ced5d3df9d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:18.598918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0cf8c68e-15b7-4ad8-8283-ced5d3df9d54 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:18.601278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0cf8c68e-15b7-4ad8-8283-ced5d3df9d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:18.653604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 68/142] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:18 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/users/b9aec71002b149c6b1beb79cdf0db364/roles/ca6f073019a043d6826e5e9ef797a451 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:18.769943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 75/143] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:19.129997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c71f-d9a0-4cb2-8f3a-8b82442a18c7 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.130888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c71f-d9a0-4cb2-8f3a-8b82442a18c7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.131333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d004c71f-d9a0-4cb2-8f3a-8b82442a18c7 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.132965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 69/144] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:19.142628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.143150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.143601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.166255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:19.167755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:19.238589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.254417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 76/145] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:19.364305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c71f-d9a0-4cb2-8f3a-8b82442a18c7 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.586998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d004c71f-d9a0-4cb2-8f3a-8b82442a18c7 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:19.667130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:19.668238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:19.668780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:19.669280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:19.696513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:19.704166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:19.720135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:19.721720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:19.724503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 70/146] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:19.732673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:19.967955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7404f299-60c3-487f-adae-7e0b0943331c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.030603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.032183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.032993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.033758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.035437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.036408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.039484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.054413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.060357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 77/147] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.069983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.199619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.325718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49935d28-5c86-41c6-be00-42bf2f81af61 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.333917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f941df6f-637c-4683-95c4-82a6bc068f34 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.406696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.407796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.408377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.408862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] PATH_INFO: `/v3/users/69904edd2b3d480d904195e97f267ebd` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.411091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] RBAC: Authorizing `identity:get_user(user_id=69904edd2b3d480d904195e97f267ebd)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.413891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.421554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 78/148] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/users/69904edd2b3d480d904195e97f267ebd => generated 289 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.423628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.424726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.425218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.425706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.432378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.460558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:20.471044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:20.474471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:20.477627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 71/149] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 420 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:20.485983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.675548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.738549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cc634f-1eb9-4c5d-8fda-2f28d07dcde9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.779733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585df55d-cce9-4320-80a8-a21a6da62427 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:20.781747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:20.834095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.834762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.835563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.835563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.836591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.838083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.839845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 79/150] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.849849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:20.862859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:20.864383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:20.865100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:20.865918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:20.872463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:20.873352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:20.876722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:20.899349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 72/151] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:20.910352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.133526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bb96c4-e529-4683-988e-ab13b3c6f0b8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.188892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaaeb59-6cdc-42ea-b507-1e916097e11e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.213690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.214461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.214744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.215028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/users/69904edd2b3d480d904195e97f267ebd/roles/7b1f222929844a9b84e1979770f32290` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.221699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, user_id=69904edd2b3d480d904195e97f267ebd, role_id=7b1f222929844a9b84e1979770f32290)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.223185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.261460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:21.285451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.286140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.286424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.286701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.287946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.289458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.291218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 73/152] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:20 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:21.300210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:21.322776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 80/153] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:20 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/users/69904edd2b3d480d904195e97f267ebd/roles/7b1f222929844a9b84e1979770f32290 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:37:21.456454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4baffcf-ce07-401a-8f08-db4268d4001d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:21.527220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:21.528386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:21.528921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:21.529411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:21.683544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:21.711268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:21.714055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:21.974606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 74/154] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 675 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:21.983943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.149161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.305636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea92b7e1-754c-47b6-b034-95aa600ac662 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.397491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.398980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.399684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.400399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] PATH_INFO: `/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.402862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] RBAC: Authorizing `identity:get_project(project_id=2f12ff72a2c14db7a56eaeb9c6fd69a7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:22.406252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e4b07-94ff-4588-8be3-bb63876daed6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:22.409767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 81/155] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:21 2022] GET /identity/v3/projects/2f12ff72a2c14db7a56eaeb9c6fd69a7 => generated 312 bytes in 426 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:22.416731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:22.661342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:22.688100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7e037b-18bb-484a-8689-fcfc57d1736d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:22.774304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:22.775310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.775777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.776281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.777627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=2f12ff72a2c14db7a56eaeb9c6fd69a7, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:22.778253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:22.781077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349d8236-b28e-4885-ae71-56a9ba31eddb None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:22.793458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 75/156] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:22 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=2f12ff72a2c14db7a56eaeb9c6fd69a7&role.id=2061abea01db4efb90f840ed1ff9465d => generated 608 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:22.960451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973d884-a9c9-4171-a629-ec272948f7b0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.961411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973d884-a9c9-4171-a629-ec272948f7b0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.962170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973d884-a9c9-4171-a629-ec272948f7b0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.967615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 82/157] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:22.971885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:22.972196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:22.972748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:22.998532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:22.999792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:23.101419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 76/158] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:23.670240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973d884-a9c9-4171-a629-ec272948f7b0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:23.932988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973d884-a9c9-4171-a629-ec272948f7b0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:23.955555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.001059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.002208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.002668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.003160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.038101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:24.054615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:24.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:24.062836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 83/159] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:24.071256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.259427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:24.298932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68907933-0c77-483e-9d98-f62f477cc958 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.391902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.392960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.393491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.394019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.394969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:24.395507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.399184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba11853-4dee-4882-a4c1-02551c6000ce None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.419687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 77/160] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.429435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.602858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c2ba-a057-471e-b57b-39e6d22b5e23 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.603042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c2ba-a057-471e-b57b-39e6d22b5e23 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.603327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817c2ba-a057-471e-b57b-39e6d22b5e23 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.605105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 78/161] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:24.613962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493603bc-9566-4518-9384-39820b80509b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.614348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493603bc-9566-4518-9384-39820b80509b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.614531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493603bc-9566-4518-9384-39820b80509b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.657707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-493603bc-9566-4518-9384-39820b80509b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:24.659810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-493603bc-9566-4518-9384-39820b80509b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:24.715670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ffeb29-5140-4e36-a3b5-4f00fe2afc15 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:24.834108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:24.835698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.836490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.837635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.840122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:24.840810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 79/162] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:24.844432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403cc8c-691f-4c44-a4b0-8ba10e347329 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.844432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403cc8c-691f-4c44-a4b0-8ba10e347329 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.844432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c403cc8c-691f-4c44-a4b0-8ba10e347329 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.847814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 80/163] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:24.848007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:24.854604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:24.854796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:24.855075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:24.858036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 84/164] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 430 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:24.869898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:24.893299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:24.894514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:24.992023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.083176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be39514c-64cf-4a65-b3e4-c7271cb2a293 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.094496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 81/165] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:25.150247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.150812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.151096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.151371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.239082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.240609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.291718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.341627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.368585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.385702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 85/166] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:25.395547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.695890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd554cf-8e0e-4332-ab5e-f3bf8afc4deb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.708434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3acc9c2-656f-4893-b9f3-365ddbc541d5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:25.741767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.742781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.743079 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.745681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.745681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:25.746638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c7cffd-dc80-4cde-8981-d1362881c071 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:25.749471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 86/167] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:25.750729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6500-4e0a-4d16-b87c-cf72459568a5 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.751013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6500-4e0a-4d16-b87c-cf72459568a5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.751294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f6500-4e0a-4d16-b87c-cf72459568a5 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.754383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 87/168] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:25.756398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d21e-9c12-45ca-879c-af7f14b58dde None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.757050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d21e-9c12-45ca-879c-af7f14b58dde None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.757623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9d21e-9c12-45ca-879c-af7f14b58dde None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.761101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 88/169] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:25.762556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d21e-9c12-45ca-879c-af7f14b58dde None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.773993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:25.774643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.774926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.775203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.786654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.789112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-dd700376-8fb2-4da9-bf5d-9c4972559381 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:25.795839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:25.803100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 82/170] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:25.805046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:25.807103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:25.807103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:25.820652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:25.821885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:25.901539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:25.912201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 83/171] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:25.913127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:25.979629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e9d21e-9c12-45ca-879c-af7f14b58dde None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.025482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.026139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.026888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.027179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.040823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-bbb5649f-897b-4445-8d30-6f2890f5ea98 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:26.047292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:26.048381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:26.049198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 89/172] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:26.049856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.050748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.050748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.074729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:26.077272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:26.159585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.165212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 90/173] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:26.166445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.176256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec5720d-81db-46ab-a5ec-7da523ffbb38 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.274833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.276011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.276605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.277178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.279380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.280032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.283246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a14f91a-8ff9-4106-956e-1ecc64b636cf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.298833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85cba60c-eec2-4b44-9093-e623cc77a3c3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.311081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 84/174] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:25 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=2061abea01db4efb90f840ed1ff9465d => generated 608 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.313693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e640f8-2fa9-4196-a737-db1823c76991 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.314614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e640f8-2fa9-4196-a737-db1823c76991 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.315270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e640f8-2fa9-4196-a737-db1823c76991 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.328163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 85/175] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:26 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:26.329841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e640f8-2fa9-4196-a737-db1823c76991 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.349144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.350225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.350751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.351240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.352251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.352812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.354916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff375d9-9a03-48cc-9fd9-dc416516cf2d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.366872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 91/176] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.368683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831896da-2a76-475b-b012-070e61324d34 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.369317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831896da-2a76-475b-b012-070e61324d34 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.369802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831896da-2a76-475b-b012-070e61324d34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.391663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-831896da-2a76-475b-b012-070e61324d34 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:26.393409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-831896da-2a76-475b-b012-070e61324d34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:26.484238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.498897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 92/177] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:26.499827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831896da-2a76-475b-b012-070e61324d34 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.517530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.570002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e640f8-2fa9-4196-a737-db1823c76991 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.634236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.636006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.636957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.637711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.640095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:26.640974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.644265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.663269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 86/178] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.665191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.706904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831896da-2a76-475b-b012-070e61324d34 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.751881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.752872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.752872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.753091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] PATH_INFO: `/v3/users/d6d84dfa12274652b8eb04c682b6e60e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.759251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] RBAC: Authorizing `identity:get_user(user_id=d6d84dfa12274652b8eb04c682b6e60e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:26.760898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c235e4c8-2629-42e6-8718-cc6174befccd None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:26.765173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 93/179] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/users/d6d84dfa12274652b8eb04c682b6e60e => generated 247 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:26.767423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816fb6-4916-469d-a7ce-49186d41bb71 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.767903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816fb6-4916-469d-a7ce-49186d41bb71 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.769964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23816fb6-4916-469d-a7ce-49186d41bb71 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.773371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 94/180] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:26 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:26.773858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23816fb6-4916-469d-a7ce-49186d41bb71 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.866965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c182c6c6-2b0a-4d40-a0b3-658b206ab107 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.876036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.908228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:26.921110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.921714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.921991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.922256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.925609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23816fb6-4916-469d-a7ce-49186d41bb71 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:26.942307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:26.952917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:26.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:26.954644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:26.955506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 87/181] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:26.956572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:26.965356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:26.966499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:26.966782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:26.967051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:26.979296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:26.987747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:26.989394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 95/182] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:26.990411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.080633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02186956-2b6f-46bf-8545-17b189d6a380 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.131637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.132646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.133138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.133616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] PATH_INFO: `/v3/users/b9aec71002b149c6b1beb79cdf0db364` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.134590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70b399-a3c6-4252-9b14-f865a3f310a2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.135619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] RBAC: Authorizing `identity:get_user(user_id=b9aec71002b149c6b1beb79cdf0db364)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.138474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e4c97a-74ca-44b1-b1f1-5beecc742a0d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.141041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 88/183] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/users/b9aec71002b149c6b1beb79cdf0db364 => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.143484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.143842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.144221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.158410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.163743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:27.165013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:27.174975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.175600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.175872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.176177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.219571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.244034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 89/184] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:27.245883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.263967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.265695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.418224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a14e7-69c9-4aee-821b-752420027402 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.441186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 96/185] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:27.442247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.470279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.492839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.493758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.494158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.494549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.496464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.500104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.503599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 90/186] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.504720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.583566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.590115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696ae46-93d0-49b1-9803-f526bcef5540 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.642140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.642879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.643241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.643614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.644635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:27.645198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.646787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.658647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fa478-4b6b-4a16-ae4b-09cce83ad72f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.660298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 97/187] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.668013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.710555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438856-cade-4425-9969-418e84789186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.711084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438856-cade-4425-9969-418e84789186 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.711366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438856-cade-4425-9969-418e84789186 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.711809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46438856-cade-4425-9969-418e84789186 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.712584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46438856-cade-4425-9969-418e84789186 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:27.713008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46438856-cade-4425-9969-418e84789186 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.714453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46438856-cade-4425-9969-418e84789186 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.724575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 91/188] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.725672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438856-cade-4425-9969-418e84789186 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.782969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.808751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5a15b-a1b7-4ada-b631-36a13c07e5f2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.847767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.848429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.848711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.848994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.862104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46438856-cade-4425-9969-418e84789186 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:27.875088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:27.913944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:27.914982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:27.915426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:27.915758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:27.917793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.920250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:27.923324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 92/189] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:27.924759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:27.953257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:27.954759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.046082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.067133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 98/190] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:28.067998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.197268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac8537c-c360-4d98-a839-29e42cb40691 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.210823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a989a49-7bb6-4aa0-ada2-1c2759586a09 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.239237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.239830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.240117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.240551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.241088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:28.242563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.266598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.267658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.268177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.268673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.288325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:28.298571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:28.299728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:28.300745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 99/191] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:28.302018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.357895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.363838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 93/192] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:27 2022] POST /identity/v3/users => generated 297 bytes in 440 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:28.365302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.422865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.452120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c134ed4-db89-496e-a20c-cfef0783726f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.511095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.511755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.512040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.512328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.533421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:28.534930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.543963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 100/193] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:28.543963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.563619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1aae0-622a-4c0b-9226-e29ee3ed93f7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.626026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.626801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.626980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.627251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.629718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:28.632083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.634213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 94/194] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:28.634213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.722540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.728070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6cb4f3-50ea-4742-92cd-ef60f250d2e2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.792455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca111a3-015f-4c40-a920-e760f2899ec5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:28.795449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:28.802546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.803556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.804125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.804769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.805849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=d6d84dfa12274652b8eb04c682b6e60e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:28.806337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:28.809999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.822316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 101/195] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/role_assignments?user.id=d6d84dfa12274652b8eb04c682b6e60e&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:28.823480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:28.873068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:28.874902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:28.878824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:28.878824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:28.878824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:28.879246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:28.884114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 95/196] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:28.887766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.074494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c537ff6-850d-4d5d-bb7e-fe2b88d48115 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.131756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.132496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.132750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.133080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.133705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.134006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.135452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.144059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd48a3c5-90ff-43a1-b8a5-7f679ab871dd None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.151853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 102/197] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.160177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.199025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.203297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.204028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.204343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.204629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.257413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.291960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.293524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.298923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c164d-bdf0-42fa-b188-7b9fd13dbb7f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.343870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.344598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.344891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.345171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.412473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 96/198] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 528 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:29.413425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.433178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.434796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.468941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.507040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.569801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440a545-0f41-46dd-a936-d93e71875b88 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.577795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 103/199] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:29.579083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.637471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.638492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.638961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.639448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.640852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=b9aec71002b149c6b1beb79cdf0db364) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:29.641402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.644118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.656181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 97/200] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/role_assignments?user.id=b9aec71002b149c6b1beb79cdf0db364&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.657860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.707265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac801f2-12be-4f63-b5c7-37435df04b61 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.748707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:29.769818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:29.771026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:29.771324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:29.771606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] PATH_INFO: `/v3/users/69904edd2b3d480d904195e97f267ebd` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:29.773092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] RBAC: Authorizing `identity:get_user(user_id=69904edd2b3d480d904195e97f267ebd)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:29.774780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:29.776722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 104/201] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/users/69904edd2b3d480d904195e97f267ebd => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:29.777748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:29.915237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11da38c-0c09-43ba-8b82-9e00c7258776 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.965029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7a01f-2082-48fa-b329-72228bd33b22 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:29.999752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.000506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.000861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.001228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.002636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.004888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.006953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 98/202] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.008035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.020903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.051702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.052884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.053497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.053871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.057044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.060893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.064706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 105/203] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:29 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.066288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.191389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.198669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08314305-9f30-4fec-883a-e0673d0c559c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.269834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.271332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.271849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.272443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.282158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6340b340-6f0e-4266-899f-6c79f056c153 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.362167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.363829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.364544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.364908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.365407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.365896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.367294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=48437872c74d4cd2af10feaf640851cb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.367866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.367971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.370801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.382230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 106/204] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=48437872c74d4cd2af10feaf640851cb&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.383756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.502217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 99/205] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:30.512998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:30.640372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ded3226-fe1e-4b02-a976-21ab41ff9a78 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.683595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.697133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.697763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.698046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.698363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.699165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=48437872c74d4cd2af10feaf640851cb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:30.699477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.704830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.709341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 107/206] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=48437872c74d4cd2af10feaf640851cb&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:30.746725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c66f36-48d7-428b-9be6-3731586e39fd None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:30.749830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:30.816067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:30.816673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:30.817044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:30.817353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:30.818698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:30.820274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699e049e-3fd6-41e6-a1d8-ee1063657b3f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:30.822356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 100/207] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:30.832328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:31.057114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2380e4b6-6b1b-420f-8da1-2ff97237e61b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:31.116610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:31.117596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.118104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.118580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.119562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, scope.project.id=9354393c91b3446a8adf9952f6651999, user.id=69904edd2b3d480d904195e97f267ebd) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:31.120135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:31.121927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e49c55-31e2-48f1-872e-81073a2c3351 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:31.129995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 108/208] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:30 2022] GET /identity/v3/role_assignments?user.id=69904edd2b3d480d904195e97f267ebd&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=7b1f222929844a9b84e1979770f32290 => generated 608 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:31.810244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d8b4-3f6c-443a-879a-2f26f6788951 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.810771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d8b4-3f6c-443a-879a-2f26f6788951 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.810771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42d8b4-3f6c-443a-879a-2f26f6788951 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.812413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 101/209] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:31.818457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:31.819105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:31.819635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:31.847315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:31.850142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:31.968420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.004343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 109/210] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:32.291329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d8b4-3f6c-443a-879a-2f26f6788951 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.418593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f42d8b4-3f6c-443a-879a-2f26f6788951 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.471340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.472469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.472965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.473448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.475769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.478672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83698e7e-4d1e-4a3e-9a0a-9dcfc38fa91f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.482891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 102/211] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:32.489070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:32.612381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372a95f0-1c97-47ba-8d18-c75307a23a4b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:32.629324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01282b-7d9c-4d25-8a96-e29834b878af None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.629817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01282b-7d9c-4d25-8a96-e29834b878af None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.630314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f01282b-7d9c-4d25-8a96-e29834b878af None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.639294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 103/212] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:32.642022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.642801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.643598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.664851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:32.665520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:32.665774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:32.666089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:32.666936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:32.668543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7108cc8-9c51-4535-8d1d-5575028d4fbf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:32.673818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:32.675427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:32.734694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:32.788210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 104/213] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:33.072230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.164481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 110/214] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:32 2022] POST /identity/v3/users => generated 297 bytes in 676 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:37:33.281051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9539d06c-9bde-4ece-835a-371f39b658fb None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.294453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472120b-1a4a-4d0a-8ca0-0466db5ebb41 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.294873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472120b-1a4a-4d0a-8ca0-0466db5ebb41 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.295054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472120b-1a4a-4d0a-8ca0-0466db5ebb41 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.296710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 111/215] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:33.304332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604900-1995-4103-ae6c-9172240cc808 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.305239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604900-1995-4103-ae6c-9172240cc808 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.305846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b604900-1995-4103-ae6c-9172240cc808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.336616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6b604900-1995-4103-ae6c-9172240cc808 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:33.339839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6b604900-1995-4103-ae6c-9172240cc808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:33.361165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.363323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.363950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.365036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.387241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:33.394461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:33.395456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:33.396311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:33.397516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 105/216] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:33.403665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.405737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.459662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.463231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 112/217] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:33.478866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5103-2f14-4614-aa02-2d81002fc1fc None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.479558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5103-2f14-4614-aa02-2d81002fc1fc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.480008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f5103-2f14-4614-aa02-2d81002fc1fc None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.481589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 113/218] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:33.487608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.487884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.488258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.518222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:33.520591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:33.590562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1768ec11-6d66-4061-8c70-5b95c3fc7f2c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.669661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 114/219] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:33.670778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.671805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.673279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.673279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.674001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:33.674601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.677158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.695078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 106/220] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:33.702446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.789950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.853816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:33.855794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db29ca5a-ece7-4254-82a2-4c8bb0d2cb0c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:33.920516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789175e-d768-4be7-a983-712c115db260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:33.921181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789175e-d768-4be7-a983-712c115db260 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:33.921465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789175e-d768-4be7-a983-712c115db260 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:33.921846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789175e-d768-4be7-a983-712c115db260 None admin] PATH_INFO: `/v3/users/b1b0aa501b6e478082245642fba6b000` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:33.954109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789175e-d768-4be7-a983-712c115db260 None admin] RBAC: Authorizing `identity:get_user(user_id=b1b0aa501b6e478082245642fba6b000)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:33.955610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789175e-d768-4be7-a983-712c115db260 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:33.957730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 115/221] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/users/b1b0aa501b6e478082245642fba6b000 => generated 297 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:33.964508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789175e-d768-4be7-a983-712c115db260 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:33.984557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.005712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6ed621-bbaf-4dca-aa1e-391d994ee764 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.074933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.075846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.077383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.077870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.098536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:34.111599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:34.112599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:34.113421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 107/222] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:34.125996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.178830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789175e-d768-4be7-a983-712c115db260 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.223637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.224765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.225282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.225763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.261453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.313656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.315554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.330766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b4da2-c67f-4e81-a118-16aa000263fc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.403843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.406244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.406741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.407212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.408824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:34.409687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.413685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.434847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 108/223] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.436744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.457738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 116/224] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 494 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:34.459416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.467086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.582351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da49a411-39ae-4313-87ac-87304f6138bc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.600073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.630874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.631839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.632336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.632790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.648006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:34.655062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:34.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:34.657249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:34.664224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 109/225] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:34.667388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.687651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5394534c-5b90-4109-a5b0-b3c28211b5a1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.733289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.734704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.735248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.736070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.738406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.740718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.742857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 117/226] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.744029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.802275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:34.849488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097ada8-99f3-46ba-9ff5-ca0580af03de None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:34.897973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:34.898635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:34.898955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:34.899228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:34.900775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:34.902749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:34.905361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 110/227] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:34.906357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:34.969520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b6901-540d-4e61-8e75-d262b478bd8b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.011672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.022833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.026623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.026623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.026623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.026623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:35.026623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.027359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.039014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 118/228] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.045176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.072343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c1d9c-f6ff-42df-985f-aebd8b8c2d5d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.118232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.119234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.119234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.119538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.121063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.124200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.125049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 111/229] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:34 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.126014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.145766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.258252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0d3b51-1481-454f-868e-d619a7f8f837 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.273068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673ac54-b1f3-4f8d-8435-8f87442b8e98 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.298234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.298934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.299222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.299497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.301040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.302193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.303832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 112/230] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.305023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.348730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.350330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.351088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.351885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.353778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=b1b0aa501b6e478082245642fba6b000) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:35.354644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.358902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.374185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 119/231] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/role_assignments?user.id=b1b0aa501b6e478082245642fba6b000&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=2061abea01db4efb90f840ed1ff9465d => generated 259 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.374185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.425828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.496641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.525971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4fa0c-6c53-4351-a0f0-24a83e700920 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.601968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e01b1-bcba-468c-be51-3d10d901549b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.645795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.646509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.646792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.647103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/users/48437872c74d4cd2af10feaf640851cb/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.653939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, user_id=48437872c74d4cd2af10feaf640851cb, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.655709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934321dd-c0d9-4244-ae4e-74300c347c18 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.679750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.681240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.684265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.684265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.685060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.688736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:35.692382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 120/232] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:35 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:35.705027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:35.754345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 113/233] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:35 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/users/48437872c74d4cd2af10feaf640851cb/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:35.899710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22e2b47-8951-47d6-a8cf-0b4272d378d2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:35.932506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:35.954880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:35.955572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:35.955881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:35.956171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/users/48437872c74d4cd2af10feaf640851cb/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:35.958792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, user_id=48437872c74d4cd2af10feaf640851cb, role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:35.960575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b8d4a-e856-4e7f-a25d-59d503b630d7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:36.088962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 121/234] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:35 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/users/48437872c74d4cd2af10feaf640851cb/roles/ca6f073019a043d6826e5e9ef797a451 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:36.344178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa92b0-a893-403c-999a-698059b38a8f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.344968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa92b0-a893-403c-999a-698059b38a8f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.344968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa92b0-a893-403c-999a-698059b38a8f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.346593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 114/235] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:36.352637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.353025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.353240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.381426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:36.382705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:36.461665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25fa4-6288-4451-8315-928197592987 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.462618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25fa4-6288-4451-8315-928197592987 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.463467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af25fa4-6288-4451-8315-928197592987 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.466752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 115/236] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:36.469798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 122/237] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:36.476521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:36.477098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:36.477652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:36.524733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:36.526002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:36.633258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 116/238] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:36.756604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:36.947119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c634718-46d2-499a-b524-f3f3827ffc91 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:36.975301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:36.979979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:37.034605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:37.035763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.036287 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.036784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.066991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:37.075978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:37.082283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:37.090024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:37.090810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:37.092786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 123/239] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:37.099122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:37.120415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb59eea1-aaa1-481a-996e-0e3c2d278866 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:37.164467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:37.165536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.165999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.166432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.167932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:37.169717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:37.172693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 117/240] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:37.178551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:37.350219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522db992-c413-464a-b30d-80f16c2dcaa8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:37.354992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888046c-4fe8-497b-bbf4-0e0e747ed39d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:37.393162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:37.394262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.394813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.395293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.396343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:37.398113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:37.416561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:37.447167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:37.448377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.448878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.449342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.450587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:37.451127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:37.454206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:37.472394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 124/241] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:37.480064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:37.553699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 118/242] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:37 2022] POST /identity/v3/users => generated 293 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:37.554564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:37.725879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9f05a6-a8dc-4257-84e4-70ecd449bbfe None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:37.814090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:37.815148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:37.815641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:37.816187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] PATH_INFO: `/v3/users/28600c40f0564e4cb98253a077e436a5` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:37.857408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] RBAC: Authorizing `identity:get_user(user_id=28600c40f0564e4cb98253a077e436a5)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:37.861803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:37.864431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 125/243] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:37 2022] GET /identity/v3/users/28600c40f0564e4cb98253a077e436a5 => generated 297 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:37.870697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.098802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a6ad0e-c4c2-4d13-9d31-6915d742ab4d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.150792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.151943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.152500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.152981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.231232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.244424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.246581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611f2c-2f6f-441b-a8a8-4837f7999db3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.375580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 119/244] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:38.383631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.435163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf4b71-f08d-4bfc-a74a-fe2c65af2490 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.436550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf4b71-f08d-4bfc-a74a-fe2c65af2490 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.437117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaf4b71-f08d-4bfc-a74a-fe2c65af2490 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.438774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 120/245] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:38.447015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.447380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.447573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.474264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:38.476055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:38.517764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.529156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012ead0-b9ff-43fb-917d-2ac8fc07a08f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.610128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 121/246] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:38.620567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.621485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.621945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.622398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.624547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.626330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b10726b-5fcc-4ea9-b17d-3c7133dbc5e2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.628305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 126/247] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:38.634826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:38.663542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:38.770950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981387f-2773-4e77-9dca-4610f85f5346 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.771483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981387f-2773-4e77-9dca-4610f85f5346 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.771758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4981387f-2773-4e77-9dca-4610f85f5346 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.773518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 127/248] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:38.777070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad23acc-a024-4f72-9139-fb17ffba9bfa None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:38.782904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.783526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.784005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.817565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:38.818471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:38.819096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:38.819378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:38.819649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:38.820423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=28600c40f0564e4cb98253a077e436a5) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:38.820729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:38.821210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:38.822190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:38.829532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 122/249] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/role_assignments?user.id=28600c40f0564e4cb98253a077e436a5&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=7b1f222929844a9b84e1979770f32290 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:38.917513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 128/250] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:38.991657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.117514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.140301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa583fb-c22e-4edc-a2ae-e66180000b68 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.187793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.188452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.188735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.189068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.205300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:39.211231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:39.212436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:39.213223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:39.214320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 123/251] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:39.220019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.343553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.363529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb3f0d-f2fa-4333-aca0-61de8b93f56c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.422139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.422787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.423068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.423345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.424043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:39.424388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.425827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.436880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 129/252] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.442841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.588722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.692933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ebc829-2f18-402b-b7a6-26aa7a462f06 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.758211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d21d82-99eb-4563-a252-ef2df3db1d19 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:39.781044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.782026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.782488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.782961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] PATH_INFO: `/v3/users/b1b0aa501b6e478082245642fba6b000` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.784489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] RBAC: Authorizing `identity:get_user(user_id=b1b0aa501b6e478082245642fba6b000)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:39.786362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef724b8c-e320-46a0-9383-b9d75b57d42a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:39.788459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 124/253] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/users/b1b0aa501b6e478082245642fba6b000 => generated 297 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:39.789972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c37aaf-a7c3-4dc3-a83f-130699f0b0c8 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.790605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c37aaf-a7c3-4dc3-a83f-130699f0b0c8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.791094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c37aaf-a7c3-4dc3-a83f-130699f0b0c8 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.800179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 125/254] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:39.801613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c37aaf-a7c3-4dc3-a83f-130699f0b0c8 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:39.831871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:39.835642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d121497-b125-4196-81a8-23609c885629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:39.837223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121497-b125-4196-81a8-23609c885629 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.837753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121497-b125-4196-81a8-23609c885629 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.838240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d121497-b125-4196-81a8-23609c885629 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.860999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d121497-b125-4196-81a8-23609c885629 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7d121497-b125-4196-81a8-23609c885629 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:39.867250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:39.868480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:37:39.869347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:39.872935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 130/255] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:39.878394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:39.878952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:39.879441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:39.910975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:39.913207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:39.917357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:40.055657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 131/256] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:40.057907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c37aaf-a7c3-4dc3-a83f-130699f0b0c8 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.062096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.142936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.143948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.149864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.150647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.152284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.153999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.156099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 126/257] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:39 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:40.171014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.297288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:40.304755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c52cafb-f1db-40f6-b171-939ae13afb4e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.351315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e31219-17cb-4d63-bbee-f950edc6dbf1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.363489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.368646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.369216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.369679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/users/b1b0aa501b6e478082245642fba6b000/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.372134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, user_id=b1b0aa501b6e478082245642fba6b000, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.374609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.433278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.433914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.434196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.434470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.435165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:40.435474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.437447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.453562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 132/258] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:40.459756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.462879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:40.481067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 127/259] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:40 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/users/b1b0aa501b6e478082245642fba6b000/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:37:40.499163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.628842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a999f1-92cf-4a5b-a2ed-31db2225b08a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.673814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.674904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.675404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.675890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.686814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:40.692527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:40.698435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:40.701198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:40.702606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 128/260] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:40.709612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.734894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6266c5e-aedd-4a79-bd2f-5b7fb59a8e95 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.845376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.846166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.846391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.846686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.848855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.854488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.854488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 133/261] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 393 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:40.860119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:40.869147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d4f00-3a2e-4b2f-bf1a-0f160aad082e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:40.946615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:40.947153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:40.947432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:40.947703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:40.948407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:40.948708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:40.950111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:40.965117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 129/262] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:40.971819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:41.052525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.053711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.106868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5d516-2528-4814-b4e5-f921c3aca75c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:41.139782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7dad4-66a2-40a2-aef8-6437de6950f3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:41.152053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:41.152754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:41.153092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:41.153437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] PATH_INFO: `/v3/users/48437872c74d4cd2af10feaf640851cb` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:41.155429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] RBAC: Authorizing `identity:get_user(user_id=48437872c74d4cd2af10feaf640851cb)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:41.156975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:41.159008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 130/263] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/users/48437872c74d4cd2af10feaf640851cb => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:41.165695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:41.258094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:41.259194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:41.259719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:41.260288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:41.302622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3445f104-8123-4aa4-95f7-0614fe53918e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:41.343518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.350798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:41.351483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:41.351773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:41.352050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:41.459278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:41.462429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:41.463179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:41.464728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:41.592811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 131/264] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:41.592811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:41.626830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.677852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.713973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f508fc-0dae-4696-8ec0-b245d4874343 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:41.756638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:41.757419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:41.757693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:41.758026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:41.759462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:41.761079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:41.768365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 132/265] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:41 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:41.769981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:41.771910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 134/266] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 912 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:41.805406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:41.902545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:41.907153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67b6c20-4b86-4d43-9442-45f1ef9a32fd None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:41.948580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:41.949446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:41.949768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:41.950105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:41.951071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=48437872c74d4cd2af10feaf640851cb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:41.951491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:41.954274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa02888-ab76-4083-b4ff-ad050ed0dc26 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:41.967052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 133/267] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:41 2022] GET /identity/v3/role_assignments?user.id=48437872c74d4cd2af10feaf640851cb&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:42.017167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7cb2-1da4-4eed-9aa0-f204553a3091 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.017557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7cb2-1da4-4eed-9aa0-f204553a3091 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.017725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db7cb2-1da4-4eed-9aa0-f204553a3091 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.024167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 134/268] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.028906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.029298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.029446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.059662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:42.079325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d43046d-4800-4430-956a-f8d4421a031b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.183062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 135/269] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:42.184243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.185070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.185467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.185853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.188037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:42.189796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:42.191832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 135/270] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:41 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 387 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:42.198449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.324773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:42.336564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6111c62c-1eb6-4cad-8f2f-bd6f84d10a3e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.381271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.381906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.382187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.382460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.383237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=48437872c74d4cd2af10feaf640851cb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:42.383545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:42.385087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c921f-6c68-49ed-8397-cf193ffbce01 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:42.396299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 136/271] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/role_assignments?user.id=48437872c74d4cd2af10feaf640851cb&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:42.756721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:42.806023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:42.836350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8429f-d9c9-4b1d-84e0-c98e80efc4b0 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.836938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8429f-d9c9-4b1d-84e0-c98e80efc4b0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.837216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8429f-d9c9-4b1d-84e0-c98e80efc4b0 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.843005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 137/272] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:42.844831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab49e-b77f-4061-ae8e-ef88cba85f77 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.845122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab49e-b77f-4061-ae8e-ef88cba85f77 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.845395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aab49e-b77f-4061-ae8e-ef88cba85f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.869804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-45aab49e-b77f-4061-ae8e-ef88cba85f77 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:42.871046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-45aab49e-b77f-4061-ae8e-ef88cba85f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:42.896992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316d00d-542d-45b5-9ab9-ecda2e6f8a5e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:42.961880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:42.963058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.963567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.964075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.987647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:42.992377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 138/273] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:42.993481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1f58-2925-4cc1-86d2-8b1728818fc1 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:42.993757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1f58-2925-4cc1-86d2-8b1728818fc1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:42.994027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63df1f58-2925-4cc1-86d2-8b1728818fc1 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:42.995700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 139/274] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:43.001354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.001631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.001920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:43.004706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:43.006535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:43.008309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:43.013297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 136/275] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:43.026765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.041185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:43.043587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:43.208506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 140/276] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:43.231278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.236371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06288ea4-12cd-463c-a1d7-a523f690da37 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.278709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.279236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.279591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.279908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.280649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:43.280956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.282450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.297563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 137/277] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:43.300104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.340845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.342689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.472169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfdedfa-1184-43e8-9f30-ebebb80deeb2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.482438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcb81df-8ce6-4a2a-a641-ac8167209b28 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.513179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.513926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.514842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.515454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.516660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.518347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.520671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 138/278] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:43.525106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.525596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.525787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.526044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.526295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] PATH_INFO: `/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.543075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] RBAC: Authorizing `identity:get_user(user_id=18d3e0174e9e4f73b5c4cb0a622cc845)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.544936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.551510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 141/279] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845 => generated 293 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:43.551510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.571991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.649584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.677861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46da75-8913-4720-9191-a5694a73e07c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.687812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07546f81-bb75-4f25-a8c1-986f9506788d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.724350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.725009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.725286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.725561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.732045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.732732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.733547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.733838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.734483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.735998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.740258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:43.745404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:43.747606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 139/280] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:43.750429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.870445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:43.885848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6c6643-d563-476f-b5fb-65b9cd02a50e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:43.895144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 142/281] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:37:43 2022] POST /identity/v3/users => generated 293 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:43.896081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:43.935261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:43.935896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.936777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.937064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.938030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, user.id=18d3e0174e9e4f73b5c4cb0a622cc845, scope.system=all) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:43.938497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:43.940251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41edec79-e2c0-4816-8a46-1b90a6677c9d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:43.947124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 140/282] 10.222.0.97 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/role_assignments?user.id=18d3e0174e9e4f73b5c4cb0a622cc845&scope.system=all&role.id=2061abea01db4efb90f840ed1ff9465d => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:43.948606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b330-2f09-4ed4-9931-90d0b793ed8f None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.949459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b330-2f09-4ed4-9931-90d0b793ed8f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.950752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2783b330-2f09-4ed4-9931-90d0b793ed8f None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.956191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 141/283] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:43.961453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:43.961749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:43.963478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:43.997052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:43.998519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:44.120179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 142/284] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:44.128006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f12a1a-f73f-4be3-8fc8-00efa9743df6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.168604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.181471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510a3c-773d-4326-923b-71890988584d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.182114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510a3c-773d-4326-923b-71890988584d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.182396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510a3c-773d-4326-923b-71890988584d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.182669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d510a3c-773d-4326-923b-71890988584d None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.183377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510a3c-773d-4326-923b-71890988584d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.183679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510a3c-773d-4326-923b-71890988584d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.185183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d510a3c-773d-4326-923b-71890988584d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.198079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 143/285] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.205015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.321263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.345022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b3220e-ad13-417a-8496-8c99331ea1a4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.386932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.387823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.388289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.388728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] PATH_INFO: `/v3/users/28600c40f0564e4cb98253a077e436a5` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.390857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] RBAC: Authorizing `identity:get_user(user_id=28600c40f0564e4cb98253a077e436a5)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.392364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.394269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 143/286] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/users/28600c40f0564e4cb98253a077e436a5 => generated 297 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.400062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510a3c-773d-4326-923b-71890988584d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.438700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.537160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d510a3c-773d-4326-923b-71890988584d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.573313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11bd830b-077e-4e8b-aedc-70feefa60aec None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.595026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.596570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.597108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.598000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.600384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.603784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.606789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 144/287] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.613137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.625617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.626668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.627152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.627643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.641727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:44.647251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:44.648279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:44.649068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 144/288] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:44.654965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.681399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.728029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:44.825671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5776d6-510d-4c84-bb70-233fa1d5c381 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.845083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c47ecd5-5216-4ee4-af1f-aacec036ab9c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:44.886630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.887253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.887534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.887806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.888600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:44.888909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.891410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:44.905782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 145/289] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:44.909307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:44.916886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:44.917778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:44.917956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:44.918109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/users/28600c40f0564e4cb98253a077e436a5/roles/7b1f222929844a9b84e1979770f32290` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:44.921636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, user_id=28600c40f0564e4cb98253a077e436a5, role_id=7b1f222929844a9b84e1979770f32290)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:44.923274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.036616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.064678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec40aac5-aee5-4366-a79a-fe14ea4ea8a2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.085617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 145/290] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:37:44 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/users/28600c40f0564e4cb98253a077e436a5/roles/7b1f222929844a9b84e1979770f32290 => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:37:45.119087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.119956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.120501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.120868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] PATH_INFO: `/v3/users/b1b0aa501b6e478082245642fba6b000` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.122652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] RBAC: Authorizing `identity:get_user(user_id=b1b0aa501b6e478082245642fba6b000)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.124897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.127539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 146/291] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:44 2022] GET /identity/v3/users/b1b0aa501b6e478082245642fba6b000 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.134774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.268787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b0a88-a848-4aec-b396-e6cc509f7199 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.310132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.313233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.313955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.314307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.314591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.401886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.403911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.558881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 146/292] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:45.568948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.734264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c989704-b6c3-498d-9b65-c2ab5e5959ea None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.785606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.786261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.786589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.786870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.788106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.789688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f278e-d1ea-491b-9cc1-c88e08a94501 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:45.791718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 147/293] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:45.798136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:45.837463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:45.953871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2606aea-7ee4-4978-b4fc-a3267d2d7c34 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:45.992841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:45.993457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:45.993735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:45.994010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:45.994747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=b1b0aa501b6e478082245642fba6b000) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:45.995056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:45.996569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a7bb9-9a48-4d54-a2c6-ca78c29b8834 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:46.004166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 147/294] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:45 2022] GET /identity/v3/role_assignments?user.id=b1b0aa501b6e478082245642fba6b000&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=2061abea01db4efb90f840ed1ff9465d => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:46.755013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565170-fd3c-4b3f-a29c-9e7833c4c6fe None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.755543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565170-fd3c-4b3f-a29c-9e7833c4c6fe None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.755618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d565170-fd3c-4b3f-a29c-9e7833c4c6fe None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.757423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 148/295] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:46.765867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be52a5-75b2-4422-8de8-c116ece9edbc None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:46.766560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be52a5-75b2-4422-8de8-c116ece9edbc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:46.767047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9be52a5-75b2-4422-8de8-c116ece9edbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:46.793399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b9be52a5-75b2-4422-8de8-c116ece9edbc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:46.795034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b9be52a5-75b2-4422-8de8-c116ece9edbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:46.887239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 148/296] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:47.171736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d565170-fd3c-4b3f-a29c-9e7833c4c6fe None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.194278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.314447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115809d-f327-4363-87a2-d696d31c03aa None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.315059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115809d-f327-4363-87a2-d696d31c03aa None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.315402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5115809d-f327-4363-87a2-d696d31c03aa None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.324584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 149/297] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:47.327696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43846-0991-495e-82fc-6c8f2f522f18 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.328825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43846-0991-495e-82fc-6c8f2f522f18 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.329328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c43846-0991-495e-82fc-6c8f2f522f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.364005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d9c43846-0991-495e-82fc-6c8f2f522f18 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:47.367415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d9c43846-0991-495e-82fc-6c8f2f522f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:47.385866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d565170-fd3c-4b3f-a29c-9e7833c4c6fe None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.392903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.476529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.479527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.480428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.480917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.506477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:47.517015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:47.518748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:47.520433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 149/298] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:47.525734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.535341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 150/299] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:47.675220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:47.754652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e51ef6d-396a-4480-9249-e06a7461eca7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:47.756897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab037-3a64-4030-abb0-18cc80a37ebc None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.757192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab037-3a64-4030-abb0-18cc80a37ebc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.757600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dab037-3a64-4030-abb0-18cc80a37ebc None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.759454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 151/300] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:47.765415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f7b-5692-43d4-ba3b-c4092599f19c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.765689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f7b-5692-43d4-ba3b-c4092599f19c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.766049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4153f7b-5692-43d4-ba3b-c4092599f19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.788335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b4153f7b-5692-43d4-ba3b-c4092599f19c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:47.789641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b4153f7b-5692-43d4-ba3b-c4092599f19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:47.823976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:47.825029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.825496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.825977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.827232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:47.827766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:47.830381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af219e-bcba-4d16-b0de-04977cdd182a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:47.846442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 150/301] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:47.848712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea039d-6030-4de6-ae3f-07eca02a66ce None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.849212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea039d-6030-4de6-ae3f-07eca02a66ce None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.849704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ea039d-6030-4de6-ae3f-07eca02a66ce None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.852051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 151/302] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:47.853578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea039d-6030-4de6-ae3f-07eca02a66ce None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:47.879431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 152/303] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:47.891265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:47.891748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:47.892275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:47.916162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:47.918894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:48.018565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 153/304] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:48.039414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ea039d-6030-4de6-ae3f-07eca02a66ce None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.090573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.097510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.101810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.113120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.114261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.114963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.115430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] PATH_INFO: `/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.117916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] RBAC: Authorizing `identity:get_user(user_id=18d3e0174e9e4f73b5c4cb0a622cc845)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.120888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.123844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 152/305] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:47 2022] GET /identity/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845 => generated 293 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.130540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.257432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf558b61-181b-441d-82db-5e8126e04f0b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.258877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8f8d7d-0db7-4382-a590-eb9f75bf436e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.307185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.307714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.308186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.308368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] PATH_INFO: `/v3/system/users/18d3e0174e9e4f73b5c4cb0a622cc845/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.309870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=18d3e0174e9e4f73b5c4cb0a622cc845, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.311278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.319365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.320101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.320623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.320974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.341635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:48.351545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:48.358932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 154/306] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:48.360348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.428324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 153/307] 10.222.0.97 () {58 vars in 1334 bytes} [Thu Sep 8 13:37:48 2022] PUT /identity/v3/system/users/18d3e0174e9e4f73b5c4cb0a622cc845/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:37:48.429458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.482675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.535594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.549377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bad1f9-a397-4f94-8e03-1bd035adb897 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.612022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.612022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.612022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.612022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.620272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:48.620530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7194c466-8b71-4a05-8b26-98f5fe5ff16e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:48.626418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:48.628277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:48.629759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 155/308] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:48.633734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.687955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.688631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.688908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.689189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.689893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:48.690202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.691670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.709662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 154/309] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.709662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.829756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e306a3f9-1630-431a-9310-eb5829a2959d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.875646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.887290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d04a7b9-84cc-4d4a-8b3b-d331832d95e5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:48.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.906114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 156/310] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.907847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:48.929989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:48.944776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:48.945448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:48.951879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:48.951879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:48.951879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:48.951879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:48.951879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:48.970017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 155/311] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:48.970017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.109423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d77142d-7c50-4ae8-9f52-b0e52cf23603 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.138452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1128463a-84ee-4624-beb4-0d09689faa4d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.168605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.169616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.170085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.170554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] PATH_INFO: `/v3/users/30acd01caff9425dbdba4529019c4e39` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.189871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] RBAC: Authorizing `identity:get_user(user_id=30acd01caff9425dbdba4529019c4e39)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.191983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.201351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 157/312] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:48 2022] GET /identity/v3/users/30acd01caff9425dbdba4529019c4e39 => generated 293 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.201351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.203670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.204399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.204676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.205110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] PATH_INFO: `/v3/groups` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.205613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.207105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.264213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.317338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 156/313] 10.222.0.97 () {60 vars in 1134 bytes} [Thu Sep 8 13:37:48 2022] POST /identity/v3/groups => generated 229 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:37:49.317338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.351094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09384d1b-2d9d-4ca3-acf0-2322213b2460 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.423007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.447641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.450249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.451195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.451930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] PATH_INFO: `/v3/users/28600c40f0564e4cb98253a077e436a5` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.455979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] RBAC: Authorizing `identity:get_user(user_id=28600c40f0564e4cb98253a077e436a5)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.459275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.463850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 158/314] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/users/28600c40f0564e4cb98253a077e436a5 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.473875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:49.539457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95775abd-4cdd-4980-a4b6-bb50152fa36c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.610839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.611809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.612310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.612592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.613488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, user.id=30acd01caff9425dbdba4529019c4e39, scope.system=all) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:49.613993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.616917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:49.628876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 157/315] 10.222.0.97 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/role_assignments?user.id=30acd01caff9425dbdba4529019c4e39&scope.system=all&role.id=7b1f222929844a9b84e1979770f32290 => generated 226 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:49.648082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fdd271-cca4-416e-b62c-261f195be005 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:49.687872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:49.723338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:49.726048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:49.727295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:49.728042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:49.884691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:49.888175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.128468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 159/316] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 651 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:37:50.134323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.323161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7759b7a-b465-42d8-bf40-1edc711e7801 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.345729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.378017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.378818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.379049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.379330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.380711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.382309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284eff05-3ec7-4256-8567-71243541fc7a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.384486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 158/317] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:50.392543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:50.436336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:50.600599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ec339-33a2-495a-a6a1-ca49f85beb99 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:50.651091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:50.651726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:50.652006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:50.652312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:50.653071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, scope.project.id=132c557d3d6b43b5b427062abf162744, user.id=28600c40f0564e4cb98253a077e436a5) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:50.653381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:50.654852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2240433-b3fb-49bb-9784-34028898ff62 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:50.663134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 160/318] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:37:50 2022] GET /identity/v3/role_assignments?user.id=28600c40f0564e4cb98253a077e436a5&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=7b1f222929844a9b84e1979770f32290 => generated 608 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:51.386789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a23d-f862-4b6c-9e03-6d54cfc8dbec None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.387515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a23d-f862-4b6c-9e03-6d54cfc8dbec None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.388072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3a23d-f862-4b6c-9e03-6d54cfc8dbec None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.390665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 159/319] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:51.397949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54af23-141d-429e-9895-bcaea9acba88 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.398464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54af23-141d-429e-9895-bcaea9acba88 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.399065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be54af23-141d-429e-9895-bcaea9acba88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.434074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-be54af23-141d-429e-9895-bcaea9acba88 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:51.435309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-be54af23-141d-429e-9895-bcaea9acba88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:51.528943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 161/320] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:51.808901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a23d-f862-4b6c-9e03-6d54cfc8dbec None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:51.966159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3a23d-f862-4b6c-9e03-6d54cfc8dbec None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:51.990273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaadc-ba1c-4f87-91ef-db60faae4872 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:51.990772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaadc-ba1c-4f87-91ef-db60faae4872 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:51.991160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25acaadc-ba1c-4f87-91ef-db60faae4872 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:51.993372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 162/321] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:52.002186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.002669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.003177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.006518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.007159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.007438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.007711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.020064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:52.025099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:52.026099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:52.029085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 160/322] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:52.033263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.036077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:52.038207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:52.053064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.088091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.150302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 163/323] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:52.162069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aec71e-fdb2-4e43-b190-8d7ae6729950 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.201644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.202276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.203257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.203540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.204520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:52.204827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.206224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.216038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 161/324] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:52.222737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.340396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b0d0b4-cde3-48b8-b8b1-feb74d0dc350 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.380610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.381266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.381547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.381820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] PATH_INFO: `/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.383056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] RBAC: Authorizing `identity:get_user(user_id=18d3e0174e9e4f73b5c4cb0a622cc845)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.384620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.386537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 164/325] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/users/18d3e0174e9e4f73b5c4cb0a622cc845 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:52.392662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.517446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.521844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945b44b4-4567-4e34-85ec-0ea7291a7660 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.563833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:52.568099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.568758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.569043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.569327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.570071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] RBAC: Adding query filter params (role.id=2061abea01db4efb90f840ed1ff9465d, user.id=18d3e0174e9e4f73b5c4cb0a622cc845, scope.system=all) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:52.570467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:52.571957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:52.579956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:52.589444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 162/326] 10.222.0.97 () {56 vars in 1308 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/role_assignments?user.id=18d3e0174e9e4f73b5c4cb0a622cc845&scope.system=all&role.id=2061abea01db4efb90f840ed1ff9465d => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:52.755376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6018d6-589a-4714-97c6-a5715fbe4f7d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:52.810769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:52.811418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:52.811689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:52.812124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:52.826451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:52.832596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:52.834298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:52.835749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 165/327] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:52.843003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.004430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea14e25-6270-416f-8a12-a9850e3eb50f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.065911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.066862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.067332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.067825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.069331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:53.069824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.072511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf30d20-9409-49d8-8b5e-b694141f73f8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.094845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 163/328] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:53.096213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.123550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a819e-d1ac-4e6f-b1c7-b2eb30f40268 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.123925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a819e-d1ac-4e6f-b1c7-b2eb30f40268 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.124342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2a819e-d1ac-4e6f-b1c7-b2eb30f40268 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.125818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 164/329] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:37:53.134869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.135276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.135728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.157674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:53.159802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:53.182748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.191538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.239463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949acc89-f821-45fe-bbc6-6c08f897ffbe None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.257594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 165/330] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:53.291981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.293136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.293597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.294115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] PATH_INFO: `/v3/users/30acd01caff9425dbdba4529019c4e39` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.296896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] RBAC: Authorizing `identity:get_user(user_id=30acd01caff9425dbdba4529019c4e39)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.299562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.307700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 166/331] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/users/30acd01caff9425dbdba4529019c4e39 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:53.309135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.470536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7479a9c9-b74e-4323-ad02-48c729d6bc79 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.512392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.513219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.513611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.513975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] PATH_INFO: `/v3/system/users/30acd01caff9425dbdba4529019c4e39/roles/7b1f222929844a9b84e1979770f32290` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.516182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=30acd01caff9425dbdba4529019c4e39, role_id=7b1f222929844a9b84e1979770f32290)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.517861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.537450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.595072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 166/332] 10.222.0.97 () {58 vars in 1334 bytes} [Thu Sep 8 13:37:53 2022] PUT /identity/v3/system/users/30acd01caff9425dbdba4529019c4e39/roles/7b1f222929844a9b84e1979770f32290 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:37:53.675798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c449db-10cb-426a-b0df-d0837b411d17 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.741112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.742157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.742618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.743075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.745197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.747778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4c6384-8e53-4c41-a62f-0ff84d60db99 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:53.750991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 167/333] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:37:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:53.757363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:53.779100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:53.908656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c2abec-0c4c-4f6b-8cda-678bf0971360 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:53.950606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:53.951226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:53.951510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:53.951784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] PATH_INFO: `/v3/groups` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:53.952623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:53.954356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27cec3e2-3076-45ca-9cf0-448eeea7f9b5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:54.083285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 167/334] 10.222.0.97 () {60 vars in 1134 bytes} [Thu Sep 8 13:37:53 2022] POST /identity/v3/groups => generated 226 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:37:57.068679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a9bf-8d87-4f49-a7d1-36204c781bcf None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.069438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a9bf-8d87-4f49-a7d1-36204c781bcf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.069438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb0a9bf-8d87-4f49-a7d1-36204c781bcf None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.071074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 168/335] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:57.077279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56baf7f-237a-4652-911a-1ae4d5acc5ad None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.078566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56baf7f-237a-4652-911a-1ae4d5acc5ad None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.078566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56baf7f-237a-4652-911a-1ae4d5acc5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.107281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b56baf7f-237a-4652-911a-1ae4d5acc5ad None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:57.108675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b56baf7f-237a-4652-911a-1ae4d5acc5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:57.214945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 168/336] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:57.394922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e45b2-9828-4d73-a5b5-b9f55ee1c3c6 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.395197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e45b2-9828-4d73-a5b5-b9f55ee1c3c6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.395680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46e45b2-9828-4d73-a5b5-b9f55ee1c3c6 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.397500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 169/337] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:57.414413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.415064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.415511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.429643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:57.446628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:57.447881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:57.523427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b464a0-8ef9-4728-a4ea-eaafef213e45 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.523918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b464a0-8ef9-4728-a4ea-eaafef213e45 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.524438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b464a0-8ef9-4728-a4ea-eaafef213e45 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.527044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 170/338] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:57.537244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b74-de27-4b15-b007-b36c0a68decf None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.537724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b74-de27-4b15-b007-b36c0a68decf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.538216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f2b74-de27-4b15-b007-b36c0a68decf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.554631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 169/339] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:57.562817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:57.574498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2e1f2b74-de27-4b15-b007-b36c0a68decf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:57.576061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2e1f2b74-de27-4b15-b007-b36c0a68decf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:57.662199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 171/340] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:37:57.793816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:57.893766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f61b2-5350-441e-8544-63460460cc19 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.894484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f61b2-5350-441e-8544-63460460cc19 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.895046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9f61b2-5350-441e-8544-63460460cc19 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.897032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 172/341] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:57.904992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.910332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.910332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.926103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cdbbf7-1ce1-419b-9526-905de430c0f7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:57.930659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:37:57.932067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:37:57.983661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:57.984407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:57.984758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:57.985038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:57.988500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:58.001598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:58.010784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:58.012980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:58.014573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 170/342] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:58.027136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:58.047208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 173/343] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:37:58.167318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f1e4f-3486-4b9b-bd5c-b16c89083efb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:58.197170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:58.258561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:58.259196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:58.259573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:58.259854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:58.260670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:58.260980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:58.264615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:58.289047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 171/344] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:37:58.298224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:58.392683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:58.469738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c61e2e-d443-4ad5-b3c7-de405981dbfa None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:58.523043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:58.586394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:58.588552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:58.589747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:58.590430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:58.614604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:58.628527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:58.630553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:37:58.632436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:58.644374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 174/345] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 440 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:37:58.646464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:58.650432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637213ab-829f-488e-9aa9-3e0389abaeee None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:58.723104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:58.723676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:58.723964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:58.724271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:58.832680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:58.834698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:58.870821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d252e638-c762-4398-95ed-0a421e9aed59 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:58.917001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:58.931501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:58.943751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:58.945146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:58.945623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:58.946085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:58.974534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:58.995004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:58.997504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 08 13:37:58.999619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:37:59.000966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 172/346] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:37:59.000966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.004999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 175/347] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:37:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:59.007157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.195884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0e44fd-6d0a-48c1-ae39-03700de7c16c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.236687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1ba73-440f-44a9-9c74-0bb2d3ecfc80 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.279373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.280602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.281643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.282159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.287388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.288391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.288647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.288915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.289614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] RBAC: Adding query filter params (name=member) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:59.289915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:59.291392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:59.304008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 176/348] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:59.304629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.311160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:37:59.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:37:59.330414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 173/349] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:37:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:37:59.333135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.374253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:59.410895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:59.540009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c37717-4cc8-451e-b034-b75263eedb96 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.570725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cd3314-f352-478e-b263-bdaf79ceaa96 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.618936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.621160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.622065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.623119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.627081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:59.631159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a21fa-6bea-49ea-83ad-48ff3d67f398 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:59.632995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.633632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.633915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.634204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.634895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] RBAC: Adding query filter params (name=reader) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:59.635203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:59.636673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:59.637305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 177/350] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:59.645007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af943-911e-47bc-80b6-a282437d8396 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.645007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af943-911e-47bc-80b6-a282437d8396 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.645380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7af943-911e-47bc-80b6-a282437d8396 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.646992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 178/351] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:37:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:37:59.647837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7af943-911e-47bc-80b6-a282437d8396 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.658262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 174/352] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:59.659064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:37:59.820341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:59.823733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7af943-911e-47bc-80b6-a282437d8396 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.855652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:37:59.867938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d528c-e22a-49fb-af14-b8e70c89d137 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:37:59.894721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.895750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.896350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.896855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.926370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:37:59.927004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:37:59.927280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:37:59.927547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:37:59.935131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:37:59.935131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:37:59.935131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:37:59.948167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 175/353] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:37:59.949777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.033863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.035373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb50ca7-1e46-45d4-83bb-c130dac7bbee None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.138961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca1202e-c13a-4a1c-867d-9f9f51680801 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:00.186441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 179/354] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 539 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:00.187793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.188075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.188380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.195270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:00.196850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.197414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.198041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.210902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.211099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:00.212293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:00.214156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.217417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 176/355] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:37:59 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:00.220907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.227160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.229973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.328692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 180/356] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:00.337245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.426607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d7d24d-b894-4278-8ba3-43a56b437b00 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:00.502052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:00.503629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.504464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.505367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] PATH_INFO: `/v3/users/30acd01caff9425dbdba4529019c4e39` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.509088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] RBAC: Authorizing `identity:get_user(user_id=30acd01caff9425dbdba4529019c4e39)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.513616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7393f195-acfd-4092-9c3a-76f0eeeac7ad None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.518086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 177/357] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/users/30acd01caff9425dbdba4529019c4e39 => generated 293 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:00.521089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a44c8-eb22-4673-9b5f-3f4cb08f9b6f None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.522444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a44c8-eb22-4673-9b5f-3f4cb08f9b6f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.523610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785a44c8-eb22-4673-9b5f-3f4cb08f9b6f None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.526373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 178/358] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:00.529318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a44c8-eb22-4673-9b5f-3f4cb08f9b6f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.530047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafaf577-9e57-4583-9aa2-2ab20436adfe None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:00.609341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:00.610753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.611353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.611840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.638624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.694539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:00.730633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.733112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.754562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785a44c8-eb22-4673-9b5f-3f4cb08f9b6f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:00.827822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:00.829371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:00.830072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:00.830526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:00.833050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:00.834794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:00.840329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 179/359] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:00.841807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.894010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 181/360] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:00.895222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:00.960341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.022400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5701a8-c1cc-4dd1-9feb-33a395348bb5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.053143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09f8cb8-9c93-475a-b63f-4082e8bec996 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.080665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.081987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.083626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.083626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.083626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=132c557d3d6b43b5b427062abf162744) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.084408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.085963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b1a591-4efa-4830-a3da-49b09c069eb4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.097424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 180/361] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=2061abea01db4efb90f840ed1ff9465d => generated 260 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.099433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.099891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.100448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.117356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.118430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.119038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.119926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.120775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.121736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] RBAC: Adding query filter params (role.id=7b1f222929844a9b84e1979770f32290, user.id=30acd01caff9425dbdba4529019c4e39, scope.system=all) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.122195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.123999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.144868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:01.147127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:01.156410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 182/362] 10.222.0.97 () {56 vars in 1308 bytes} [Thu Sep 8 13:38:00 2022] GET /identity/v3/role_assignments?user.id=30acd01caff9425dbdba4529019c4e39&scope.system=all&role.id=7b1f222929844a9b84e1979770f32290 => generated 510 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.156410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.324325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 181/363] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:01.325144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.386338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f73cacc-73ad-49a6-ad75-926abb6d1f34 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.393313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.488580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.489671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.490156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.490640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.514577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:01.529686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:01.530650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:01.531709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:01.532931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 183/364] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 381 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:01.533671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.567697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c426faf7-b19e-41ec-b37a-4188c9c7acae None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.644673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.645717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.646216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.646686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.648771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.651453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.655936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 182/365] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.657439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.670773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.777870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e24c1-baa1-47d7-82d1-44f41994ea13 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.853908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.855566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.855890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.856202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.857585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.860233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.862024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 184/366] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.862843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:01.866183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ae1438-263f-41a8-9ae4-cadbbe86e640 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:01.925256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:01.946145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:01.948429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:01.948895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:01.949296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:01.950226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:01.950749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:01.953896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:01.971377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 183/367] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:01.973808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.065899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500f9d25-da06-4608-a485-8feebe0117c2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.095180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.136286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.137738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.138512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.139262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.141354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=9354393c91b3446a8adf9952f6651999) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:02.141980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.144608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.153629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 185/368] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=2061abea01db4efb90f840ed1ff9465d => generated 260 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:02.160209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.164294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b7414c-2fe0-46e8-82a8-33f465a45aad None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.233885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.235198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.235836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.236347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.259610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:02.273411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:02.275611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 184/369] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:02.277753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.288957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.302642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9037c40-9817-4ea3-b526-52aeca42a371 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.380432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.382538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.382538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.382900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.385469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.388449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.392381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 186/370] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:02.394113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.418970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f54485-97e7-44a9-b9aa-3c314eff9320 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.447646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.461500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.462201 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.462488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.462782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.548630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.550212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.577521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d8945f-0b3b-42f1-a14b-d5842d274373 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.617416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.618029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.618308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.618579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.619254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:02.619553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.621031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.631746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 187/371] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:02.632661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.670025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 185/372] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:02.670968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.672620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.702882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:02.787488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077475c-a306-4ae8-be9e-3ecefa210f5e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.829613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.830726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.830856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513df84-f491-4340-9431-dd03c22fb490 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:02.831292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.831727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:02.832776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=9354393c91b3446a8adf9952f6651999) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:02.833260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:02.834917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:02.842315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 188/373] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:02.843549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:02.893271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:02.895316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:02.896914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:02.897202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.047494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:03.070631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c7f4c-4481-4f70-b60f-c2dfcec2d8a3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:03.075558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.077501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.108082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:03.165639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:03.170396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:03.171027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:03.171529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.173738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.175554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.178558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 189/374] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:03.185097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:03.225899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 186/375] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:03.232553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:03.362807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57dce0-63a3-45ee-a25f-5935fcfc6049 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:03.408398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e820e5-13a1-4fe7-be77-5e18e637661a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:03.448595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:03.456522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:03.457805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:03.458343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:03.458824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.460648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.463290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.465593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 190/376] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:03 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:03.473345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:03.488131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:03.488813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:03.489095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:03.489373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] PATH_INFO: `/v3/projects/5225debdf3f04c89923ec01541ea8b94` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.491470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=5225debdf3f04c89923ec01541ea8b94)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.495394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.500781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 187/377] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:03 2022] GET /identity/v3/projects/5225debdf3f04c89923ec01541ea8b94 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:03.506544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:03.539912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:03.662964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c584bc90-e257-4c79-9cf9-71b63bbaa6a6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:03.705572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:03.706234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:03.706515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:03.707290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] PATH_INFO: `/v3/groups/43a0f6238ac14851b456664ec9b61db7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.713213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] RBAC: Authorizing `identity:get_group(group_id=43a0f6238ac14851b456664ec9b61db7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.714979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.717315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 188/378] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:03 2022] GET /identity/v3/groups/43a0f6238ac14851b456664ec9b61db7 => generated 229 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:03.730736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:03.770615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57bb8e-70f5-4f51-8a53-d7c50b19b55b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:03.883802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:03.884470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:03.884746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:03.885025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:03.885782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=132c557d3d6b43b5b427062abf162744) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:03.886092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:03.887577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde63ff5-1e16-42ec-aee6-031f76c94b58 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:03.903808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 191/379] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:03 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 260 bytes in 431 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:03.925111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:04.051384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feca4fe9-6ca3-4db4-bbc5-4988ebff974c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:04.142158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:04.143253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.143745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.144281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.145690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] RBAC: Adding query filter params (group.id=43a0f6238ac14851b456664ec9b61db7, role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5225debdf3f04c89923ec01541ea8b94) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:04.146267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:04.149037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403c56f-c139-4087-ac32-f45e29357a07 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:04.165225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 189/380] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:03 2022] GET /identity/v3/role_assignments?group.id=43a0f6238ac14851b456664ec9b61db7&scope.project.id=5225debdf3f04c89923ec01541ea8b94&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 260 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:04.417181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda29e-eecf-4aae-9bd4-871f45be26bb None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.418019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda29e-eecf-4aae-9bd4-871f45be26bb None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.418019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffda29e-eecf-4aae-9bd4-871f45be26bb None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.419155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 192/381] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:04.430682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ceea-2cd4-45d7-9b02-dc8dd2474afa None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:04.431176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ceea-2cd4-45d7-9b02-dc8dd2474afa None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:04.431678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a942ceea-2cd4-45d7-9b02-dc8dd2474afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:04.476095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a942ceea-2cd4-45d7-9b02-dc8dd2474afa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:04.478391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a942ceea-2cd4-45d7-9b02-dc8dd2474afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:04.628697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 190/382] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:05.134320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffda29e-eecf-4aae-9bd4-871f45be26bb None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.168581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83367b9-9224-4f1a-81a0-9317536b1bfc None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.168940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83367b9-9224-4f1a-81a0-9317536b1bfc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.169442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83367b9-9224-4f1a-81a0-9317536b1bfc None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.171153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 191/383] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:05.178084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.178416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.178673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.213401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:05.215123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:05.319087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.349603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffda29e-eecf-4aae-9bd4-871f45be26bb None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.361124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.402614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 192/384] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:05.440283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.441135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.441419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.441694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.469860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:05.475449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:05.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:05.478134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 193/385] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:05.482995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.619338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0dfa9c-0f3f-43fd-a0e8-dce3d82252e5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.677186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:05.678154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.678636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.679159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.680164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:05.680697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:05.682388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac564af-b1f2-4135-a9aa-d88c84715f66 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:05.694114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 193/386] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:05.700590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:05.817989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9492e-8303-48b4-9681-97ad4ec4e740 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.819460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9492e-8303-48b4-9681-97ad4ec4e740 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.821526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e9492e-8303-48b4-9681-97ad4ec4e740 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.823265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 194/387] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:05.842489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:05.843062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:05.843569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:05.875055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:05.877915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:05.933238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c72a6b6-9d56-442d-aec2-d47344184f5a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:05.936656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:05.991821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 195/388] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:05.993624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.007910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.008704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.008991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.009257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.012214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.013810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.015989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 194/389] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.022370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.066655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.243253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9131e605-ecd2-4c56-953e-dad86befc438 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.322332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.323033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.323271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.323428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.325992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64224870-8042-47ec-8c19-100c214ecca5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.357965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:06.363944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:06.365800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:06.368220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 196/390] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:06.376510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.418111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.420376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.424369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.424837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.427880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.429407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.439763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 195/391] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.442152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.472358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.630318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee5cb9-e7d4-456a-8cd6-128fbed8c456 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.676978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.697784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899fe089-9022-46e6-8b6a-0a504002ccf6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.699144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.700276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.700782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.701282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.702551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:06.703105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.706629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.732893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 197/392] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:06.734781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.746785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.747902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.748419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.748891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/groups/5754bce6b628450298e942fa76153cb1/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:06.751951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, group_id=5754bce6b628450298e942fa76153cb1, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:06.753843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:06.852483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 196/393] 10.222.0.97 () {58 vars in 1442 bytes} [Thu Sep 8 13:38:06 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/groups/5754bce6b628450298e942fa76153cb1/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:06.856227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:06.928501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:06.939159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91999bcc-eb1e-4897-b25c-e4689d63c9b3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:06.998140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:06.998911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:06.999200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:06.999498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.017264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3ddc78fb-7e27-4a76-9499-8355091d1736 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:07.024762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:07.025807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:07.026585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:07.027200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 198/394] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:07.032183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb35d01-91f1-47ce-ac10-6b188d8eeffb None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.032183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb35d01-91f1-47ce-ac10-6b188d8eeffb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.032183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb35d01-91f1-47ce-ac10-6b188d8eeffb None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.035910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 199/395] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:07.036934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb35d01-91f1-47ce-ac10-6b188d8eeffb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.037648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88cc9a43-48e2-41f3-b9b6-c2f8cd8ea90f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.085585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.086643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.087201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.087673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.089198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.090779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf6b0c6-2857-4d6b-9cc4-9df7408ec1ef None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.094034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 197/396] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:06 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.095018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.095018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.095286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.119350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:07.120679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:07.133019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.208625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb35d01-91f1-47ce-ac10-6b188d8eeffb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.209109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 198/397] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:07.210208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.271619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.272305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.272589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.272855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.273544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:07.273851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.275358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a714c-8ad1-488d-b652-797a5a4fbb7c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.290906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 200/398] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.290906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72517cba-97ae-492a-b709-34f38ce97e1e None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.290906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72517cba-97ae-492a-b709-34f38ce97e1e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.290906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72517cba-97ae-492a-b709-34f38ce97e1e None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.294508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 201/399] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:07.295910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.296303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.296543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.322943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:07.324203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:07.338794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.377792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e99ad45-20b0-4f46-a9ea-814a89d3f042 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.417265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.418317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.418433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.418675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.419923 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.421493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.425211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 202/400] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:07.426524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.434555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 199/401] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.435662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.453788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.567357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc0d3b-6dee-46b2-9c6e-b1bd99e962f9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.600722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc9b8d7-f0a4-4a77-9964-ffc8b2b44be4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.611915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.613208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.613208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.613402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/groups/5754bce6b628450298e942fa76153cb1/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.615625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, group_id=5754bce6b628450298e942fa76153cb1, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.617268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.647030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.647856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.648310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.648672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.650010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.651568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.653569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 203/402] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.656558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.677605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.704329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 200/403] 10.222.0.97 () {58 vars in 1442 bytes} [Thu Sep 8 13:38:07 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/groups/5754bce6b628450298e942fa76153cb1/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:07.705718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.819957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c49b7d-cc7f-4896-9268-94e3c75824a8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.829424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:07.868975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658f6f82-dc42-496f-8a24-ed248f0b221e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:07.883692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.885216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.885695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.889845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.904538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:07.912652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:07.913871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:07.915075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:07.915909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:07.916729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:07.921686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:07.921686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:07.921907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 204/404] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:07.921907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:07.922057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:07.922057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:07.923874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 201/405] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:07.924865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.043192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.059407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbd57fa-d62e-4f17-8d2f-619f4f217c18 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.077313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27a681b-9c27-42e3-8ebf-f6c922c4da2a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.109527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.111866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.112190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.112455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.113165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.113469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.114908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.128724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 202/406] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.132466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.133496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.134008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.134113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.134763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.137341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.152335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:08.159041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:08.160048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:08.160891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 205/407] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:08.161560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.278679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80387c2-6bc5-48e5-8a97-6a3b1f14a309 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.331446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.332041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.332368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.332682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999/groups/5754bce6b628450298e942fa76153cb1/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.335049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] RBAC: Authorizing `identity:create_grant(project_id=9354393c91b3446a8adf9952f6651999, group_id=5754bce6b628450298e942fa76153cb1, role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.336732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.378108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9f616b-7519-4c28-9a38-24521ccba4b9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.450371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.451436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.451904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.452424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.455322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.458328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.462203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 206/408] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.467800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.472014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.486134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 203/409] 10.222.0.97 () {58 vars in 1442 bytes} [Thu Sep 8 13:38:08 2022] PUT /identity/v3/projects/9354393c91b3446a8adf9952f6651999/groups/5754bce6b628450298e942fa76153cb1/roles/ca6f073019a043d6826e5e9ef797a451 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:08.488102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.665771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9c7b2a-13a7-4c81-bcca-f050c6338f01 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.715674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.716341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.716837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.717190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.717906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:08.718212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.721050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.737802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 207/410] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.740654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:08.749096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.860843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bea77d-34c7-4770-a17d-ac79f1e9056f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.874304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:08.954156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:08.955141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:08.955814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:08.956311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:08.961270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:08.963828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:08.972114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ed427-7b87-45a4-8bd9-1731394e3e47 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:08.973304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 204/411] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 486 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:08.979972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.045541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.046196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.046479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.046753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] PATH_INFO: `/v3/groups/43a0f6238ac14851b456664ec9b61db7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.050268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] RBAC: Authorizing `identity:get_group(group_id=43a0f6238ac14851b456664ec9b61db7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.051961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.054002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 208/412] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:08 2022] GET /identity/v3/groups/43a0f6238ac14851b456664ec9b61db7 => generated 229 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:09.060846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.244335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47aaf71-09b3-4dbf-8586-10b2c1abcba4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.309408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.317996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd20771-9426-4892-a4f7-54cdb77afa25 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.337009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.337711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.338058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.338292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744/groups/5754bce6b628450298e942fa76153cb1/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.341478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] RBAC: Authorizing `identity:create_grant(project_id=132c557d3d6b43b5b427062abf162744, group_id=5754bce6b628450298e942fa76153cb1, role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.344374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28019318-8ccf-48a0-88f2-3fe49f3c7efd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.373149 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:09.379152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.380178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.380645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.381089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] PATH_INFO: `/v3/projects/5225debdf3f04c89923ec01541ea8b94` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.382680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] RBAC: Authorizing `identity:get_project(project_id=5225debdf3f04c89923ec01541ea8b94)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.384706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edcafda-7b86-40b2-8128-910a6e378986 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.386760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 209/413] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:09 2022] GET /identity/v3/projects/5225debdf3f04c89923ec01541ea8b94 => generated 344 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:09.388362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a634237-6aca-42c0-8cf6-8e49378ee175 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.388843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a634237-6aca-42c0-8cf6-8e49378ee175 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.389333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a634237-6aca-42c0-8cf6-8e49378ee175 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.396195 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 210/414] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:09.397406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a634237-6aca-42c0-8cf6-8e49378ee175 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:09.434598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 205/415] 10.222.0.97 () {58 vars in 1442 bytes} [Thu Sep 8 13:38:08 2022] PUT /identity/v3/projects/132c557d3d6b43b5b427062abf162744/groups/5754bce6b628450298e942fa76153cb1/roles/ca6f073019a043d6826e5e9ef797a451 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:38:09.438950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.439644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.440549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.485726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:09.489468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:09.619164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a634237-6aca-42c0-8cf6-8e49378ee175 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:09.682208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 206/416] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:09.682402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:09.683199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:09.683517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:09.683813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] PATH_INFO: `/v3/projects/5225debdf3f04c89923ec01541ea8b94/groups/43a0f6238ac14851b456664ec9b61db7/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:09.688613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] RBAC: Authorizing `identity:create_grant(project_id=5225debdf3f04c89923ec01541ea8b94, group_id=43a0f6238ac14851b456664ec9b61db7, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:09.688613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:09.767435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 211/417] 10.222.0.97 () {58 vars in 1442 bytes} [Thu Sep 8 13:38:09 2022] PUT /identity/v3/projects/5225debdf3f04c89923ec01541ea8b94/groups/43a0f6238ac14851b456664ec9b61db7/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:10.278027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.475442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb5b137-8e9a-4ffe-9c4e-3574c09f4588 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.514409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.544774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.545331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.545614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.545895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.566609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:10.581793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:10.586301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:10.587505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 207/418] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:10.595571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:10.718351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:10.788180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddad41b-8963-46e1-a5ab-bea9e0eca36d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:10.833785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:10.834311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:10.834587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:10.834839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:10.835532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:10.835832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:10.837315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7cb9ec-8540-434b-96e3-79b935dc53b1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:10.852857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 212/419] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:10.865383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.045272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be2a76-3ffc-4437-9b78-1726dd03c76b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.107001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.107541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.107824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.108076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.209526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.241429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.243661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.254268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c726-ba96-4ecc-9185-5d342de833b1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.254676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c726-ba96-4ecc-9185-5d342de833b1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.255100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce14c726-ba96-4ecc-9185-5d342de833b1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.260765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 213/420] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:11.266942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ebb3-9af9-4826-a9b2-7aa0071d0ba7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.267347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ebb3-9af9-4826-a9b2-7aa0071d0ba7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.267583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ebb3-9af9-4826-a9b2-7aa0071d0ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.305751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9d12ebb3-9af9-4826-a9b2-7aa0071d0ba7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:11.306998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9d12ebb3-9af9-4826-a9b2-7aa0071d0ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:11.468476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 208/421] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 602 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:11.476371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.490854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 214/422] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:11.614707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.644996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107ad5b-5c47-4abe-ba5d-da0ec3772190 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.700348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.701001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.701276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.701552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.702896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.705058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.708102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 209/423] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:11.715216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa8c54-c51c-409d-96ef-b786e02134c0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.715499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa8c54-c51c-409d-96ef-b786e02134c0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.715813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa8c54-c51c-409d-96ef-b786e02134c0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.717113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 215/424] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:11.718262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:11.724285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.725207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.725207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.765268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:11.767052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:11.856630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001d3c12-30ef-4225-babd-66f8654cca0c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:11.860907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:11.901966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 216/425] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:11.908459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:11.908459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:11.908459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:11.908459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:11.909936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:11.911647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:11.914128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 210/426] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:11.922244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.034311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.068029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4402ddf-2630-4e22-a414-b2128533997a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.084133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.185392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.186450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.186949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.187429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.188932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=132c557d3d6b43b5b427062abf162744) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:12.189481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:12.192345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:12.207212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 217/427] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:11 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=2061abea01db4efb90f840ed1ff9465d => generated 611 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:12.296312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae4f76e-c89f-43e4-b315-b1162bb854fd None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.357893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.358552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.358833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.359119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] PATH_INFO: `/v3/roles/member` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.380625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:12.394429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:12.396193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:12.398475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 211/428] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:12.405126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.459948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.523090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.642864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd6f80-d570-481f-8039-a00f7cbe9dc5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.670713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.698280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.699205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.699694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.700068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.700922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] RBAC: Adding query filter params (name=member) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:12.701232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:12.703003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:12.723283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 218/429] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:12.728930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5c0cab-7c0e-4c77-b842-a3ea5c9a3d15 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.731957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.801201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.802736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.803292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.803828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:12.830048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:12.836998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:12.838783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:12.840397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 212/430] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 379 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:12.845733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:12.910351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22838c5-84e6-4e5b-97a7-becc864aa036 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:12.929438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:12.970272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:12.970998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:12.971281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:12.971551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.029378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467158fe-c1ae-4e5e-a8b4-ab200442264c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.087417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.090364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa306653-35e7-41a7-a1fc-f45e8555b67b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.102723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.103774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.104334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.104842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.106271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:13.106882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.109599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db28da39-e4c3-4697-bca3-afecb75f30f8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.114442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.132249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 213/431] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:13.147248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45473b7-76b2-4195-a9ba-e65c3be7e5aa None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.147248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45473b7-76b2-4195-a9ba-e65c3be7e5aa None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.147248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45473b7-76b2-4195-a9ba-e65c3be7e5aa None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.147248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 214/432] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:13.147248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45473b7-76b2-4195-a9ba-e65c3be7e5aa None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.254085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 219/433] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:13.255390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf385-2076-4b2f-878c-5b14a81c16b1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.255730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf385-2076-4b2f-878c-5b14a81c16b1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.256009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3cf385-2076-4b2f-878c-5b14a81c16b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.275409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-eb3cf385-2076-4b2f-878c-5b14a81c16b1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:13.276671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-eb3cf385-2076-4b2f-878c-5b14a81c16b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:13.279092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.314829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45473b7-76b2-4195-a9ba-e65c3be7e5aa None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.372392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.373003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.373301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.373640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.389901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 220/434] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:13.392184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159ca4-6f95-4b77-bf8b-12d882f3abfd None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.392640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159ca4-6f95-4b77-bf8b-12d882f3abfd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.393319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b159ca4-6f95-4b77-bf8b-12d882f3abfd None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.395575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 221/435] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:13.396300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b159ca4-6f95-4b77-bf8b-12d882f3abfd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.480780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.489521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.494523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.600634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b159ca4-6f95-4b77-bf8b-12d882f3abfd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.643905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.644619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.645025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.645470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.646933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.648841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1557dac4-6a89-4665-b24d-2b1c33f88558 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.659883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 222/436] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:13.659883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.659883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.659883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.673303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.680755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:13.682443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:13.697189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 215/437] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:13.697189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.779655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 223/438] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:13.780508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.866293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:13.871546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368d8dee-35a7-4876-9f64-163e499bb34c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:13.915275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:13.916070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:13.916386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:13.916665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:13.917953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:13.919636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:13.921371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 216/439] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:13.922188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:13.964309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c863b2-45bd-4b24-a098-3b047b25fe4c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.017809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.018461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.018742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.019397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] PATH_INFO: `/v3/projects/9354393c91b3446a8adf9952f6651999` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.020320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9354393c91b3446a8adf9952f6651999)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.021889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.031442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 224/440] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/projects/9354393c91b3446a8adf9952f6651999 => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.031442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.034117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.099112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd2b38b6-cd9e-40a7-b3a8-1ccadc0b48df None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.166740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.167698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.168237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.168725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.190129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed54734-3426-4d3b-9c04-9632208e98c4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.190629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:14.200512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:14.202190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:14.203531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 217/441] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:14.205433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.249418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.250324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.250759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.251688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.253178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=2061abea01db4efb90f840ed1ff9465d, scope.project.id=9354393c91b3446a8adf9952f6651999) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.253646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.255521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.259376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.276177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 225/442] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=2061abea01db4efb90f840ed1ff9465d => generated 611 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.276177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.412517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.417443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716daa5e-d48a-40b5-83fb-8c538c077675 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.465475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573f58cf-a127-4eb7-b4ce-71a66859c446 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.471485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.472428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.472822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.473215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.475033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.477316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.488552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 218/443] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.488552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.531723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.533333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.533605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.533881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.534685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.534990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.537294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.549253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 226/444] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.550122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.644967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.654744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31198a0-3477-48e6-ae9e-74da2e6a445d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.714697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.714697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.715388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.715388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.720990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9faf5f8-f2fc-4f06-97f4-ca919623b6cc None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.735259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:14.740294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:14.741882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 219/445] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:14.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.784257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:14.792728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61830a56-30f8-4100-aec4-38473320011b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.793720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830a56-30f8-4100-aec4-38473320011b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.794211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830a56-30f8-4100-aec4-38473320011b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.794710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61830a56-30f8-4100-aec4-38473320011b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:14.802194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61830a56-30f8-4100-aec4-38473320011b None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=9354393c91b3446a8adf9952f6651999) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:14.802194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61830a56-30f8-4100-aec4-38473320011b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:14.802194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61830a56-30f8-4100-aec4-38473320011b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:14.807963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 227/446] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=9354393c91b3446a8adf9952f6651999&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 611 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:14.809170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61830a56-30f8-4100-aec4-38473320011b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:14.926514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327cc308-b211-44ff-ae9f-68020235df56 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.975505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61830a56-30f8-4100-aec4-38473320011b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:14.986140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:14.987013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:14.987400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:14.987788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.030375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.060759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.061397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.061674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.062022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.062796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:15.063207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.065771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.076857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 228/447] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:15.085010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.108979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.110998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.176383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.247811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e07eb-5dff-4014-a8b4-9a2743961378 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.292978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 220/448] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:15.300101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.310642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.311273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.311550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.311823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.389283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.424694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.426255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.460369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a79a1-274f-4862-ac82-8c3ea29834be None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.507801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.520031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.525278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.525278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.525278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] PATH_INFO: `/v3/projects/132c557d3d6b43b5b427062abf162744` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.525278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] RBAC: Authorizing `identity:get_project(project_id=132c557d3d6b43b5b427062abf162744)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.525278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.526082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 221/449] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/projects/132c557d3d6b43b5b427062abf162744 => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:15.533541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.573212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 229/450] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:15.582314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.667945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fe0a18-e295-45f2-b22f-0f3404d99c19 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.703591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.719063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.720246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.720772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.721053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] PATH_INFO: `/v3/groups/5754bce6b628450298e942fa76153cb1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.722547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] RBAC: Authorizing `identity:get_group(group_id=5754bce6b628450298e942fa76153cb1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.724515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.726464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 222/451] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/groups/5754bce6b628450298e942fa76153cb1 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:15.735539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.742078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc03115-afde-4b77-953a-6ef6c81a82be None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.791027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.791797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.792123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.792517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] PATH_INFO: `/v3/projects/5225debdf3f04c89923ec01541ea8b94` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.793812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] RBAC: Authorizing `identity:get_project(project_id=5225debdf3f04c89923ec01541ea8b94)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.795285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.797310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 230/452] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/projects/5225debdf3f04c89923ec01541ea8b94 => generated 344 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:15.802976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:15.835029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:15.886978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d9a57-2efc-44ba-9521-464ed8cdbdda None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:15.927024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:15.927665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:15.928106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:15.928619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:15.929853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] RBAC: Adding query filter params (group.id=5754bce6b628450298e942fa76153cb1, role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=132c557d3d6b43b5b427062abf162744) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:15.930357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:15.932803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:15.941507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 223/453] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/role_assignments?group.id=5754bce6b628450298e942fa76153cb1&scope.project.id=132c557d3d6b43b5b427062abf162744&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 611 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:15.956667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b880ce-d751-4c7d-bef0-7b103209cb22 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.019160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.020452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.020826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.021329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] PATH_INFO: `/v3/groups/43a0f6238ac14851b456664ec9b61db7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.023684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] RBAC: Authorizing `identity:get_group(group_id=43a0f6238ac14851b456664ec9b61db7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.026648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c833cd11-d2c4-4345-bd59-baedb0912f54 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.029950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 231/454] 10.222.0.97 () {56 vars in 1180 bytes} [Thu Sep 8 13:38:15 2022] GET /identity/v3/groups/43a0f6238ac14851b456664ec9b61db7 => generated 229 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:16.039492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:16.151535 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:16.217737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c4b39e-55b4-4a14-88f0-f8c9e53ba190 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:16.264207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:16.264844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:16.265140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:16.265407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:16.266166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] RBAC: Adding query filter params (group.id=43a0f6238ac14851b456664ec9b61db7, role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5225debdf3f04c89923ec01541ea8b94) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:16.266469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:16.268582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-118fe05b-315b-432e-9612-dd008026da64 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:16.276111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 224/455] 10.222.0.97 () {56 vars in 1376 bytes} [Thu Sep 8 13:38:16 2022] GET /identity/v3/role_assignments?group.id=43a0f6238ac14851b456664ec9b61db7&scope.project.id=5225debdf3f04c89923ec01541ea8b94&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 611 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:20.260179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510ca6-18da-4086-9a55-0b67548c0d31 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.260179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510ca6-18da-4086-9a55-0b67548c0d31 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.260179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1510ca6-18da-4086-9a55-0b67548c0d31 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.263923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 232/456] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:20.274235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a21a9-5642-4c7c-8b3d-25036749654d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.275678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a21a9-5642-4c7c-8b3d-25036749654d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.276433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a21a9-5642-4c7c-8b3d-25036749654d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.314750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b37a21a9-5642-4c7c-8b3d-25036749654d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:20.317004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b37a21a9-5642-4c7c-8b3d-25036749654d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:20.397696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176ac-ab8a-4899-847c-f22e612deae7 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.397959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176ac-ab8a-4899-847c-f22e612deae7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.398245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b176ac-ab8a-4899-847c-f22e612deae7 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.399940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 233/457] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:20.406328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f8e1-515f-44ff-a156-7ee0862da361 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.406939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f8e1-515f-44ff-a156-7ee0862da361 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.407408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b95f8e1-515f-44ff-a156-7ee0862da361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.430547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6b95f8e1-515f-44ff-a156-7ee0862da361 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:20.432192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6b95f8e1-515f-44ff-a156-7ee0862da361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:20.522918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 234/458] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:20.535312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 225/459] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:20.764042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae701396-c5ec-492f-9fb1-13251b3f612a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.764785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae701396-c5ec-492f-9fb1-13251b3f612a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.765278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae701396-c5ec-492f-9fb1-13251b3f612a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.767792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 235/460] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:20.776085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a7f5e-c08e-4df5-a137-09e557908a4f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:20.776523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a7f5e-c08e-4df5-a137-09e557908a4f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:20.776995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a7f5e-c08e-4df5-a137-09e557908a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:20.818019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3d0a7f5e-c08e-4df5-a137-09e557908a4f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:20.820017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3d0a7f5e-c08e-4df5-a137-09e557908a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:20.924321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:20.937316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae701396-c5ec-492f-9fb1-13251b3f612a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:20.976486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 226/461] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:21.010964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.021748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81671-08ff-42e0-bfbb-86e2fb47616f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.022245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81671-08ff-42e0-bfbb-86e2fb47616f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.022735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c81671-08ff-42e0-bfbb-86e2fb47616f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.025341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 227/462] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:21 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:21.033313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.034081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.034759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.067708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:21.070338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:21.178783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae701396-c5ec-492f-9fb1-13251b3f612a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.202542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 228/463] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:21.203996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.266516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.267930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.269493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.271785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.296846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.300307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:21.358579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:21.360351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:21.362211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:21.362211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.362211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:21.362211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 236/464] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 425 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:21.362440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.414417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c12fed2-3d02-4b2e-b589-f02ec1b3e80b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.440351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.505058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.506412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.507226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.507878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.509678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.527013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.529241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.549683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de7178-229c-4533-a1a7-32ab1dadf9fa None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:21.553613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:21.555108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:21.557056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:21.557056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.557056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:21.560265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 229/465] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:21.562269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.622851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.623536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.623812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.624333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.640901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.644179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:21.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:21.666369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:21.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:21.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:21.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:21.669704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 237/466] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:21.671397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.744180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dd893b-992e-42e3-8eef-6c31b1662d18 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.803476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.804304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.804728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.805132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.806001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.806412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.808070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.830555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 230/467] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.830555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.865498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.867231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9696c8a-005f-4e76-a731-0a5ac93814ad None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:21.925995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:21.940893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:21.941404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:21.941913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:21.942419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:21.943645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:21.944402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:21.947083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:21.964202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 238/468] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:21.965087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:21.995990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aad7478-3462-4eb3-84ca-b29b23fd0da2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.040485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.041406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.041847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.042292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.043352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:22.043844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.045512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.059414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 231/469] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:22.060188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.123423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce19237-29ac-4957-b738-801ff0d31614 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.175876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.177214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.177652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.178092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.186914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.195929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.198534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.218168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf38ce0c-f134-4ffa-88a3-a29934245cc3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:22.227766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:22.229948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:22.231921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:22.231921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:22.231921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:22.231921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 239/470] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:22.231921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.281840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.282767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.283257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.283695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.284838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.287252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374179f9-d622-4a56-a00e-359e6750b7bf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.296506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.371451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef174d2f-abf4-49f7-b540-9e963710767f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.397782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 232/471] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/users => generated 248 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:22.399277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c15cd-9141-4d82-92a3-7f29546285bf None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.399842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c15cd-9141-4d82-92a3-7f29546285bf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.400371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3c15cd-9141-4d82-92a3-7f29546285bf None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.404209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 233/472] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:22.404209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c15cd-9141-4d82-92a3-7f29546285bf None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.416792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.417490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.417769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.418045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.418769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.420279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.516909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 240/473] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/users => generated 246 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:22.518641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:22.602200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3c15cd-9141-4d82-92a3-7f29546285bf None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.685960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.686997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.687474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.687910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.689774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.692567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.708590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.732046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:22.743765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e351a6b-101a-408a-9a48-9075bc56d14e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:22.800008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:22.800798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.801087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.801358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.802046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:22.802348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:22.803765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e1083-e21e-4c6e-9776-b4d705a20c74 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:22.818688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 241/474] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:22.820073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:22.820450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:22.820694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:22.849358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:22.851517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:22.868528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 234/475] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/users => generated 248 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:22.870115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.031296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 242/476] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:23.152291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02c4030-0bf4-4481-92e4-9060dce1c8a8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.245590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.246708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.247181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.247637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.248824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.251787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.383630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.386151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 235/477] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:22 2022] POST /identity/v3/users => generated 249 bytes in 517 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:23.577500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.612884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75049e4-2213-4236-8094-6dd72844fb80 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:23.698023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:23.699189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:23.699668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:23.700166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:23.701464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:23.704290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:23.767103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.sql.core [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:23.767103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:38:23.767103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [parameters: {'id': '3dcd4752d7964eb9be6834a092f0c143', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:38:23.767103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88421) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:38:23.796919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return ref.to_dict() Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:38:23.799922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application yield resource Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise self.value Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application session.commit() Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:38:23.803660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:38:23.807184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:38:23.810702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:38:23.817649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [parameters: {'id': '3dcd4752d7964eb9be6834a092f0c143', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:23.821970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 08 13:38:23.825345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:23.828528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 243/478] 10.222.0.97 () {60 vars in 1131 bytes} [Thu Sep 8 13:38:23 2022] POST /identity/v3/roles => generated 144 bytes in 416 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:23.828839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:23.882704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:23.949075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5786686-10c9-4938-8f85-e820ec1cf00a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.005772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.006846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.007316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.007804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.035759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-78d32f2a-14f9-405e-a6bf-448c0b1e4415 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:24.050409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:24.052730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 08 13:38:24.054721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:24.059215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 236/479] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:38:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:24.059443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:24.199163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d87c35-2e87-4244-9dc1-6c2cc751ab31 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:24.243816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:24.244493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:24.244775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:24.245046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:24.245786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:24.246231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:24.247833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961f51e-172e-4eb0-b83c-1860f20a1d83 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:24.259000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 244/480] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:25.753140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335a0d6-679e-471a-a974-6a80ec23d664 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.754165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335a0d6-679e-471a-a974-6a80ec23d664 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.754685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b335a0d6-679e-471a-a974-6a80ec23d664 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.756642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 237/481] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:25.762859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.763372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.763578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.788183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:25.789509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:25.793268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:25.884059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880d3f9-e076-467c-84ae-eaa61d595cb6 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.884729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880d3f9-e076-467c-84ae-eaa61d595cb6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.885251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f880d3f9-e076-467c-84ae-eaa61d595cb6 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.887266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 238/482] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:25.893526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:25.893828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:25.894067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:25.895130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 245/483] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:25.929339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:25.930578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:26.103166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 239/484] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:26.200913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.364208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b6562e-5e62-4ff4-991c-e2349e77fb71 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.404226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.405351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.406005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.406522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.416661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.420126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:26.439735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:26.443026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:26.443820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 246/485] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:26.450880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.569415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.646408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.659475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c768047-a468-4060-ada3-7e2698fa516e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.722835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.723503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.723787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.724062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.724871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:26.725220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:26.726679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7668ce-100f-421b-a118-c00a9dba933f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:26.752638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 240/486] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:26.755309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c36a4e-a981-4214-bf68-e6c32797b4ce None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.755886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c36a4e-a981-4214-bf68-e6c32797b4ce None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.756294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c36a4e-a981-4214-bf68-e6c32797b4ce None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.759557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 241/487] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:26 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:26.765147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c36a4e-a981-4214-bf68-e6c32797b4ce None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:26.809206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c55d67-dd8e-4603-ac79-e59fce01fa82 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.853006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:26.853659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.853943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.854217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.867445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-919da903-837a-4768-8a30-c39a2f906bf2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:26.872586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:26.873817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:26.874714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 247/488] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:26.875391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:26.875866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:26.876035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:26.903874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:26.905128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:26.975004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.980892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:26.987178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c36a4e-a981-4214-bf68-e6c32797b4ce None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:26.996473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 248/489] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:26.997519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.049639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.051456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.053820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.054654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.078131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.079715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-4afd0701-d45e-4f39-9cc3-879d311f1c47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:27.118312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.119611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:27.120826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:27.120826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.120826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:27.120990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc95af-d3f9-40dc-8421-bfffbbeab720 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.124943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 242/490] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.128005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb75825-667d-47f2-bb36-c89cce290d37 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.128707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb75825-667d-47f2-bb36-c89cce290d37 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.129229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb75825-667d-47f2-bb36-c89cce290d37 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.132404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 243/491] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.134036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb75825-667d-47f2-bb36-c89cce290d37 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.173070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.173768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.174726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.175131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.175925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.176859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.177894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed6f24d-2c61-483e-8f60-deeec8b9fd3a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.192970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 249/492] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:27.192970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.192970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.192970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.211138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:27.212572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:27.289585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.326774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 250/493] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:27.328046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.353104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb75825-667d-47f2-bb36-c89cce290d37 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.418721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.420389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.421201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.421905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.423356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.424690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.427963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.450949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 244/494] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:27.452616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.481671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.507778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8414f46d-0013-495e-9f4b-cc0a76c282a9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.558489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.559397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.559824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.560276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.573498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.575193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-5b233c0c-385e-470f-905e-9d7a672dfc54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:27.592933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.593968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:27.594897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:27.594897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:27.594897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:27.596509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 251/495] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.598072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5472-614f-49d0-aafb-f5cc1c2af2d5 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.598356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5472-614f-49d0-aafb-f5cc1c2af2d5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.598622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d5472-614f-49d0-aafb-f5cc1c2af2d5 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.605448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 252/496] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:27.605448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d5472-614f-49d0-aafb-f5cc1c2af2d5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.647527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bccb1-8d0f-402c-b669-2be93621e697 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.650654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.711190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.712191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.712702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.713193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.726275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d5472-614f-49d0-aafb-f5cc1c2af2d5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.733281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:27.743570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.744596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:27.745368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:27.745984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 245/497] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.747112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.770360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.771029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.771300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.771696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.780537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.782318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-689398d2-5639-477b-a86b-6e2da75b9371 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:27.800899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:27.803019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 253/498] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:27.805209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.806724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.806868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.838347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:27.840544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:27.852187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:27.889063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292193ee-b7ae-4827-9b68-2171830bac0d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:27.929207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:27.929790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:27.930082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:27.930472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:27.931005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:27.931303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:27.932881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:27.946415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 246/499] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:27.947379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:27.965306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 254/500] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:27.966098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.014585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.079979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94aec587-fe33-4a86-b9db-020b38fc6956 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.136047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.138881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab136f-f882-4ecf-b9ff-18775a65dc5f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.143995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.144674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.145055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.145292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.161212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:28.168443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.170166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 247/501] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.171800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.195839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.196556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.196839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.197117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.197836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.198223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.199631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.210480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 255/502] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.211432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.335836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e3a179-5ac6-45bb-850d-9db1e46a2ebb None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.355868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.378314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.378915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894aba81-888a-4e93-8199-6e7451ed087c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.379019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.379196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.379470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.380258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.380570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.382082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.429847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 248/503] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.429847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.436106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.437183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.437672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.438148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.452475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.455866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:28.478615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.479825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:28.481540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 256/504] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.483729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.494067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.557055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f6b8d8-1b94-484e-97a4-c4ff103c54c5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.600546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.601078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.601361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.601631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.602330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:28.602638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.604104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.614739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 249/505] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:28.615730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.647585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ab384-9c7d-493d-bd30-9e9f72768f40 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.686960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.688043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.688574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.689067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.711117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.712956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.721632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:28.735142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.740656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:28.742321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:28.742321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.742321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:28.742321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 257/506] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.742321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.756840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bfdeed-1a86-4a4c-8246-6e5f48aedf9a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.774548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:28.804079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.804743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.805020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.805296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.818330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:28.819965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:28.839870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:28.841792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:28.843292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:28.843292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:28.843292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:28.844337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 250/507] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:28.845620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:28.901367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e59c8a-5178-4845-9ddf-744843a30b71 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:28.945552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:28.946174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:28.946445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:28.946726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:28.987077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1719061d-f9b5-47ff-9327-1d1266f2a619 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.028217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.029210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.029892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.030168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.030435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.031182 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.031488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.032419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.032994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.033920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.059045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 251/508] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.060448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.099372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.170643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 258/509] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:29.171894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.208279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d416ae-429b-4917-85bf-97aa924b5bd7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.245886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.246533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.246803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.247075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.259045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.260560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:29.276245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:29.278104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:29.279715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:29.279715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:29.279715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:29.279972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.282233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 252/510] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:29.283035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.284178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935779b6-b6b6-4ef1-8050-603b4ad01a36 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.325388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.326018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.326294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.326566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.327280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.327528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.328945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.343932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 259/511] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.345279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.406144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.436845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6ad39f-ce7d-4619-894d-acf50e3a42a5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.475224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.475834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.476104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.476412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.477058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.477356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.478263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8bfbe0-8260-4e4b-9ac4-6ac2d5a9c74a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.478741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.493546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 253/512] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:29.494434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.527306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.528197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.528605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.529012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.562745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.619720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.622312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.646592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f473dd3-d70c-414a-bb40-c74c32f7371b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.690718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.691271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.691548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.691831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.694384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.711024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.722749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:29.736292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:29.737605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:29.738659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:29.738659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:29.738659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:29.738659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 254/513] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:29.738886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.760626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 260/514] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:29.762078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.836983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:29.885688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74570d1-acc6-4b03-b7a5-a5e721010c03 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.925693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbde5d2-2a9c-4ecb-a0e6-c6fcbbef72b0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:29.926028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.926740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.926740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.927016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.936237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.937697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:29.959958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:29.961477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:29.962803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 261/515] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:29.963907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:29.980939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:29.982076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:29.982561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:29.983008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:29.984122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:29.984668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:29.987700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.004082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 255/516] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.005150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.060377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.102203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c79327f-6ef1-484a-ab93-05d9be9c0230 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.121912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.141142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.141764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.142044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.142315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.143600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.145204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.145403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752c63eb-9a96-4c3b-ae1c-5a70d801112c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.187200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.187816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.188115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.188402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.189058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.189357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.190860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.204008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 256/517] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.204912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.262489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 262/518] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:29 2022] POST /identity/v3/users => generated 247 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:30.264449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.382591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eb8b46-083e-4f51-8548-400f52121943 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.398465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.442760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76774704-e143-48b0-8103-84be728ee639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.443518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774704-e143-48b0-8103-84be728ee639 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.443809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774704-e143-48b0-8103-84be728ee639 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.444188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76774704-e143-48b0-8103-84be728ee639 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.460869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76774704-e143-48b0-8103-84be728ee639 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.462987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76774704-e143-48b0-8103-84be728ee639 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.478036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34e9bd-d9c5-41c7-9f40-f612dab1bfca None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-76774704-e143-48b0-8103-84be728ee639 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:30.486240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:30.488048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:30.489787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:30.489787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:30.489787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:30.489787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 257/519] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:30.489787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76774704-e143-48b0-8103-84be728ee639 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.541147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.541987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.542417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.542696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.543596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.543959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.546349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.591835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 263/520] 10.222.0.97 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.593069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.631778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.652970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76774704-e143-48b0-8103-84be728ee639 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.693954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.694592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.694869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.695141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.704364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.706205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:30.720506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:30.721983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:30.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 258/521] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:30.723420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.744196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f001712-d5a6-4857-be22-d51bfaf01ecb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.760979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:30.803918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.804691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.805105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.805916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.817794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.821023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:30.845552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:30.847127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:30.848616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:30.855595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 264/522] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:30.857027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.887255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fb8812-32a2-4a0e-bd83-e751bea75997 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:30.941903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:30.942588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:30.942868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:30.943169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:30.943831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:30.944136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:30.945620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:30.967051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 259/523] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:30.967051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:30.995853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.006521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef1b6a-24ce-4311-b6ae-b6dd43bc3e57 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.057494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.058056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.058335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.058615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.100412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.127463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1352b70-5441-40e1-9415-882d6fb1e51b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.151898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.153436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.179437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.181809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.183010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.183757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.185406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.185964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.188525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.222790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 260/524] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:31.222790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.295954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.313892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 265/525] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:31.313892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.377917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52480cad-534b-4761-9a10-4da555c2d8f2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.428192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.429119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.429594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.430011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.430904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:31.431373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.433069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.438409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.455846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 261/526] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:31.460000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.497714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d3a12-0640-47e7-88d2-8272e1fcee34 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.549554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.550689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.550979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.551254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.559684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.561741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:31.575645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:31.576816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:31.577652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 266/527] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:31.578119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.605074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e2be3a-c7c4-4bea-b8d1-11030faca819 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.649040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.661553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.662454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.662955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.666311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.717796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070cd62-c58a-4e99-9045-560cbf1bd5b0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:31.770811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:31.771893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:31.772428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:31.772923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:31.777863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:31.785726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.787231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.799862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:31.802428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:31.831690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:31.833023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:31.834211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:31.834211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:31.834211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:31.834211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 267/528] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:31.835024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.940867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 262/529] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:31.940867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:31.996087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742f64b6-209e-4ddb-9501-f14f1d179843 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.002216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.050587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.051200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.051595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.051871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.052734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:32.053143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.054628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.072188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 268/530] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.072188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.127372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.137091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7622d5-5d1b-481f-8328-0caec3315567 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.211021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.212562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.213087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.213946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.216014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:32.216747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.219166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339b055b-dd96-43a5-917d-7da18d1b730e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.219603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.252929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 263/531] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.254374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.283056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.285389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.285853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.286310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.287419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:32.288888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.291901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.324382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 269/532] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.325398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.380278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.449077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d65202-93dd-435d-a9ab-e95ea3c98f32 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.485083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be5f2b8-1076-4aad-a3cb-c15c8c75a15c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.529917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.531521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.532038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.532563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.558254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.559645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.560161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.560270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.560673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.562083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=eb8f5ef6eb4948edbf947eacffea57bb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:32.562651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.562848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.565717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.584743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 270/533] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/role_assignments?user.id=eb8f5ef6eb4948edbf947eacffea57bb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.586457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:32.594529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:32.595589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:32.597089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:32.597089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:32.597089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:32.601994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 264/534] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:32.603657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.638976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.780168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f420513-1b32-45eb-9978-0a17f6b95dc7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.792626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:32.823870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.824578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.824861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.825139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.835083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.836884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.838624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33454e01-9e6d-47b1-b2fa-f401a898a165 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-8fdec734-c0bc-40b0-a105-8579850474c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:32.853041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:32.854758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:32.856505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 271/535] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:32.857588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7374b4-50a1-483c-92a2-3ba006627496 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.857588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7374b4-50a1-483c-92a2-3ba006627496 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.857588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7374b4-50a1-483c-92a2-3ba006627496 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.864304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 272/536] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:32.868586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7374b4-50a1-483c-92a2-3ba006627496 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:32.944312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:32.945864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.947221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.947905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:32.949910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:32.950683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:32.954568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6781bd-bba8-4810-9e48-db2c404fbbd1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:32.985171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 265/537] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:32.988309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:32.989055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:32.989791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.030644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:33.033445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:33.076024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7374b4-50a1-483c-92a2-3ba006627496 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.150662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.155466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.156637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.157129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.157672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.158061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 266/538] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:33.160015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.160196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.160727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.163030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.202346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 273/539] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.212487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.246696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.334949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50bec9c-7fde-4160-b72f-b13b2a46f813 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.426268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.432175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.432175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.432175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.444368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19ffbce-d170-48fb-85e3-b8b2f5f0dc30 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.447502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.449043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:33.474744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:33.475726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:33.476600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 267/540] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:33.485260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.569478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.574164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.574987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.575372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.575750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.576596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=f87892da36634672bac2c4c12a52f15e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.576977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.578561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.594404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 274/541] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/role_assignments?user.id=f87892da36634672bac2c4c12a52f15e&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:33.637056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afa68a0-ceba-40ec-a345-1c9442b488e1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.696298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.697354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.697938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.698424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.699381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.699981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.701750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:33.726093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 268/542] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:33.736446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.739755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:33.893044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.918510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068d857f-3a79-4266-849a-fb0485d58de3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.924303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3802585-1230-443d-9399-39e90cb38eed None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:33.957772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:33.980975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:33.982765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:33.984069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:33.984591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:33.985781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=9f7790a1b25740fba9cb92c1fa53ba2e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:33.988220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:33.991725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.003667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 275/543] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/role_assignments?user.id=9f7790a1b25740fba9cb92c1fa53ba2e&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.004391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.005870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.006396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.006929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.020739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:34.026597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.027573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:34.028402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:34.031439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 269/544] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:34.040881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.212364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0cede1-265f-488a-9fa9-74cc9cb8f8b7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.281443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.282440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.282947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.283433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.285174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.285697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.288347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.322387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 276/545] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.331406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.520712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb4eede-560b-4627-926d-d42bafe9abe0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.566679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.567423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.567732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.568012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.581756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.583288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:34.606271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:34.607902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:34.608929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:34.608929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:34.608929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:34.608929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 270/546] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:34.617935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:34.626529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:34.887608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4794815b-0c40-44c1-9528-212d04c40df9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:34.956743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:34.957771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:34.959339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:34.960081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:34.965357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:34.965968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:34.967700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:34.988726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 277/547] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:34.997107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.042658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.136080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3f2ad2-7806-48e7-b680-8ac6e9a31bda None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.185734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.186336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.186621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.186898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.196848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.198455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-35660199-202d-4fc8-b49d-03192f75dbdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:35.212655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:35.213739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:35.214802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 271/548] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:35.223366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.338151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.393240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb0da9f-25dc-4a2d-b61e-4bb6bc5b69f0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.458321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.459644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.460164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.460660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.461657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:35.462196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.463868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.474361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e70d1-834c-467b-93c0-d148d702d245 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.474909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e70d1-834c-467b-93c0-d148d702d245 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.474909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e70d1-834c-467b-93c0-d148d702d245 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.476503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 272/549] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:35.489636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.489912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.490196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.492574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 278/550] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:35.501124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:35.538254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:35.539485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:35.701802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 273/551] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:35.725313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a720acf0-f3f4-471d-89d6-6e3b0bc4957f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:35.754171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:35.778167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:35.778799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:35.779147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:35.779428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:35.863817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:35.868055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:35.985301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 279/552] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:38:35.994370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.021471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.056743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.105623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.138560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41802d1-595b-48ad-af7a-b0bc8f5d0e02 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.187351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.188099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.188510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.188872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.202297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.203862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-6caed9e1-2aa3-456e-8e62-844b69ba1685 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:36.223243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.226742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 274/553] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a399-22ae-4f8d-aade-8a579bab24dc None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a399-22ae-4f8d-aade-8a579bab24dc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.230190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d976a399-22ae-4f8d-aade-8a579bab24dc None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.238343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 275/554] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:36.243431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.246756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.246756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.279545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:36.280117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83442368-e6ca-4875-8f26-f1e545cd5ddc None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.281413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:36.354932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.356700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.357617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.358424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.380754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:36.391839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.394276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:36.396087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:36.408612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 280/555] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 345 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:36.411067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.424367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 276/556] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:36.429969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.443399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.621441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.642680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3469cb-0ba0-4798-a7ee-89284d98d3d6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.684087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.685005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.685386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.685794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.686556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.686962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.688677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b305f9dc-7fdf-4ab9-8edd-9cc0c1e8dca7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.698545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56a9b70-2796-422f-97a2-9fbc656f9e25 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.705528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 281/557] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.707052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bffb38-334b-4b2f-b2b9-714b1602756b None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.707647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bffb38-334b-4b2f-b2b9-714b1602756b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.708315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bffb38-334b-4b2f-b2b9-714b1602756b None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.715695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 282/558] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:36.716989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bffb38-334b-4b2f-b2b9-714b1602756b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.752675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.753724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.754215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.754709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.755863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:36.756423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.758964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fa9684-ff81-49ba-8e10-817fba412965 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.774740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 277/559] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:36.777026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.777517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.778017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.807224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:36.809251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:36.845296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bffb38-334b-4b2f-b2b9-714b1602756b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:36.898061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:36.898647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:36.899535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:36.899823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:36.907766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:36.910253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:36.910529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.913881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:36.924083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:36.925947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 283/560] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:36.928696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:36.933318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 278/561] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:36.934110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.090692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c0cc74-4532-4e77-9e5d-a8da11325e0d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.102553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc98e24-3a9d-4a36-88b0-02a4c7a6ddff None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.145493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.146802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.147302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.147955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.162241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.163554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.164132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.164724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.165655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.167141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.182386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:37.183778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.184781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:37.186130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:37.186130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:37.186130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:37.186130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 279/562] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.186130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:37.193156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.194218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 284/563] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.195863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.254426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.270488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.319268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e979331-578f-43b9-8afa-77f3efb9a526 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.336771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df911435-7ce2-4800-8675-c12b05f3534b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.357768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.358377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.358646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.358912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.359602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.359906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.361482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.381439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 280/564] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.381439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.406930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.407561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.407840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.408113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.408815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.409146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.410599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.430545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 285/565] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.432186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.540786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffee2ce-638d-4f33-9653-eb330a1205d4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.554971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.597472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.598595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.599066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.599487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.600507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.600992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.603135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.625290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 281/566] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.625290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.625554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09002004-42e1-4b88-972f-1b8d4c58be54 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.642568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.660819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.676385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.676385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.676385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.676385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:37.680937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.682824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 286/567] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.684509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.770220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f4277b-efcd-49d8-87cc-9eb74244e3c2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.812445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.813202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.813567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.813968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.814831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=71620f9e7c3147829e384a5cfae55f07) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:37.815246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.817101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a9612-1398-4fbb-9e2b-6fe19109b1b3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.817292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047b281-d7a1-424a-a322-6b7b2dd319c3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:37.824196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 282/568] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/role_assignments?user.id=71620f9e7c3147829e384a5cfae55f07&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:37.825696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86acc29-7aa6-428f-a764-fe52447c4ca6 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.826150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86acc29-7aa6-428f-a764-fe52447c4ca6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.826600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86acc29-7aa6-428f-a764-fe52447c4ca6 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.832772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 283/569] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:37.835022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86acc29-7aa6-428f-a764-fe52447c4ca6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.866571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:37.867258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:37.867538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:37.871003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] PATH_INFO: `/v3/users/nova` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:37.877647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:37.879199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:37.883825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:37.892684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:37.902337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 287/570] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/users/nova => generated 82 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:37.904316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:37.961886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.006001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86acc29-7aa6-428f-a764-fe52447c4ca6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.045814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bffc4e-eb8e-4239-a5e9-25ca6c6f1aef None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.057327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.058076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.058421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.058791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.072249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.073790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-68d0eefb-5a87-4959-b194-a18e20231c1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:38.091221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:38.092606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:38.093508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:38.093508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.093508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:38.093508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 284/571] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:38.094179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.094461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.094796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.098840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.099887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.100444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.100980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.102914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:38.103400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.105117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.121441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:38.122825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 288/572] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:38.128429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:38.128634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.240605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 285/573] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:38.241960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.272586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.286737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be0175e-601b-4608-860a-84828bac1656 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.290357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.364914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.368113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.368795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.369442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.371949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:38.372825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.375801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.424387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 289/574] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:38.426715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.480557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3765925-9475-4da9-811a-5c9dc711e19d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.551771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.552740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.553218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.553707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.554860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:38.555393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.558039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.581643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 286/575] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:38.583205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.629939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86259d01-a334-46eb-a7a1-e994d62035b3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.677440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.678316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.678794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.679436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.700963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.704249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.718588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:38.731461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:38.733463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:38.735438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:38.735438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.735438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:38.735438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 290/576] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:38.735438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.764936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7fd68f-b3d2-4639-b6e8-e4a0e4ff19ee None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.770768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:38.822978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.824003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.824528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.825008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:38.846299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:38.849215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:38.879421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:38.881127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:38.882840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:38.882840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:38.882840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:38.882840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 287/577] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:38.882840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:38.912559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb68875c-cce6-4878-967e-c21f7a8b4427 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:38.977637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:38.979071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:38.979566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:38.980051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.001617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:39.011870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.013027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:39.013870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:39.016589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 291/578] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:39.018334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.065756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fb59b3-b400-4e4c-a3a9-8a2d69c2fbf4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.121377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.124584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.141114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.151665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.151665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.151665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.157545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.159014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.162952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82143d39-590e-4864-a591-d8243dddd328 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:39.173899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.184056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 288/579] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:39.186444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.244966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.245955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.246420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.246879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.248904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.249413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.251905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.279374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 292/580] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.279374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.344835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a50f24-5537-497b-96ed-7d6e4958fd6b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.399695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.400713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.401208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.401733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.402944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.403486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.406129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.430688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 289/581] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.432235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.478960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b61d83-410f-408d-ad57-edb17f90b604 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.512058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.520484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.529187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.530242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.534940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.534940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.534940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.534940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.535683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.556537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 293/582] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.556537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.620928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df94b21-30e0-4758-9377-82d4f5fcc59b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.676776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.677360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.677636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.679192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.679192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:39.679192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.680467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.709594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 290/583] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:39.709594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.739684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2c3bd1-5cb1-4d33-b8a1-eebcaa504ea2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.795886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.796734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.797115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.797491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.806615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.808324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:39.822307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.824623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 294/584] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/users/neutron => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:39.835477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.862967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca84c7a-c567-4ec4-9c68-ca8bfcb35587 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:39.868927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.905390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:39.906033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:39.906316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:39.906598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:39.914396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:39.914774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:39.916588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:39.934250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:39.935193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:39.936189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:39.940185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 291/585] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:39.941188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:39.971212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e543c432-3028-4eea-ad51-51b6bdf4a69b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.018584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.019848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.020130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.020448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.035360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.037363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:40.053630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.059663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:40.060708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:40.060708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.060708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:40.060708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 295/586] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.060708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.091616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf85f20-037e-4025-8e08-29e48b7acdff None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.132977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.133664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.133945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.134219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.146354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.147806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:40.164358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.165724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:40.167489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:40.167489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.167489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:40.167489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 292/587] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.167489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.187260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a22b70-89b9-4ee8-a98f-eb03c4ef8615 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.189310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.195186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.236674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.237450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.237729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.237995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.238748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.239050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.240680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.279741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 296/588] 10.222.0.97 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.279741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.323413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a49344-96db-42f6-9624-e861026d9c0d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.375301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.376277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.376548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.376817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.377537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.377845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.380896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.400504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 293/589] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.402144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.480507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05171a6d-ecea-4280-b7ff-e78cde9cdf40 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.509775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.538784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c65deca-77f0-46d6-a1d4-22638bd17f78 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.541242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.543739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.544134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.544451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.545180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.545649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.548097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.575217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 297/590] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.575217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.586003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.587132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.587710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.588346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.589254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:40.589649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.591051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.605628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 294/591] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:40.606556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.616158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.713640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7c97f-116e-4927-a983-c015a0af0fc5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.732122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bd4226-fa86-4dde-b873-e2137ecf3741 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.754267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.754910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.755188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.755461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.767360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.768993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.771053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:40.771759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.772059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.772687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.776436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-2c0ce07e-5c59-44dd-94c4-b94980dfb2ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:40.789851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:40.792040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:40.794141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:40.794141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:40.794141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:40.795613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=46beefbd50f34b43b82977125f31ffeb, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:40.797147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:40.797751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 298/592] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:40.799197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7620-5925-471f-8764-dd394ed563c9 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:40.799587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7620-5925-471f-8764-dd394ed563c9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:40.800562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ecd7620-5925-471f-8764-dd394ed563c9 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:40.805024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 299/593] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:40.805024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7620-5925-471f-8764-dd394ed563c9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:40.897602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:40.924418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ecd7620-5925-471f-8764-dd394ed563c9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:40.927711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 295/594] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:40 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:40.929225 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.040000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.040942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.041506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.041820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.052468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.054590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:41.071319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.073183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:41.075075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:41.081306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 300/595] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/users/glance => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.082796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.137869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2af5f01-2f64-4ae4-8688-fb8935ef246b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.193209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.194203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.194646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.195079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.207502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.210152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-2bb13d33-8203-4564-8513-c90b79bcda68 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:41.230836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.231806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:41.233113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:41.236596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 296/596] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.239680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.240526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.241139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.269823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:41.271619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:41.278005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac623b7c-21f1-4559-87da-2aa21b2eca33 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.338969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.339595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.339873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.340192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.340976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.341276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.342826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.345954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.348016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.363110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 301/597] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.369251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.384775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 297/598] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:41.385798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.559815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542403c3-4527-4610-9ffd-c6cc3d7f6076 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.594388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badd91e-4e0f-4968-bb80-af5adacb5dc6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.646983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.648236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.648827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.649232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.650672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.651225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.655245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.676736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.677679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.677964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.678240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.678967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:41.679272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.680837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.691827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 302/599] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.693597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.704298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 298/600] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:41.704298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.742754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.818775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:41.820370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603dabcd-6f3f-4f94-8619-65c1687ee06b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.865966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.866536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.866924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.867156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.883874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:41.893763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.896024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 299/601] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.898001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:41.908662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430b281f-f84e-4906-9cca-3c3ebda51a20 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:41.955323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:41.955964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:41.956447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:41.956726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:41.967054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:41.968286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:41.984339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:41.986019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 303/602] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:41.989350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.055104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.062301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af2e250-4d32-44e5-b4c4-9b405be7d292 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.120791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.121364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.121688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.121930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.134856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.136422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.150591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f48d94-8297-49ca-b325-6d5c7c809426 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:42.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.155526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:42.157329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:42.157329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:42.157329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:42.157329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 300/603] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:42.157329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.178724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.203830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.204660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.205000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.205354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/eb8f5ef6eb4948edbf947eacffea57bb/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.230685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=eb8f5ef6eb4948edbf947eacffea57bb, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.234281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.306396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdad426-dff9-4bad-bd61-a19e1544d463 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.351843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.354196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.354196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.354196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.354983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.355502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.358109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.375963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 304/604] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:41 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/eb8f5ef6eb4948edbf947eacffea57bb/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:42.377807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.383131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 301/605] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.385155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.410147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.586833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547e2460-f504-4c96-8494-219190c2cc2b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.587098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45473d40-0a90-4e98-99ff-3330d47c17c1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.645704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.647074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.647357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.647635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.648340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.648649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.650132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.657973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.659142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.659956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.660465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.661348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:42.661798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.663444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.665746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 302/606] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.666748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.687754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 305/607] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:42.689591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:42.737060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.833005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:42.881872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f0df11-13fb-4335-b96f-9b1af01e2c6d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.901410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2142ba1c-2734-49b6-bc51-ccc0a114afd6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:42.947881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.948935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.949417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.949891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.963293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:42.966026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:42.978725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:42.980247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:42.980788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:42.981269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:42.991988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:42.993050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:42.993945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:42.994366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 306/608] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:43.000020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.013462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.015033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:43.036170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:43.037195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:43.038454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:43.038454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:43.038454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:43.038454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 303/609] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:43.038933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.160644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb52c8-c12a-41a9-b640-94efdcdc41c4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.187565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.190693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.200292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.203887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.224613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 304/610] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.225045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c61714-1dd4-4dba-948a-296463bee517 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.225818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.293770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.295532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.296437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.297243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/f87892da36634672bac2c4c12a52f15e/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.334030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=f87892da36634672bac2c4c12a52f15e, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.337512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.343030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d166a9-3cf4-43f6-bc19-bc966c2e0be6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.385906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.386537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.386834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.387167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.387752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:43.388054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.389512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.408276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 305/611] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:43.408276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.474712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.505046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 307/612] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:42 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/f87892da36634672bac2c4c12a52f15e/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:43.506520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.624075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839132ef-94cb-4f4b-8718-00d14eda2969 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.700338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.703346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.704469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.705003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/9f7790a1b25740fba9cb92c1fa53ba2e/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.706171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a029034-7b68-42fc-83be-fce5f22571ca None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:43.747783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=9f7790a1b25740fba9cb92c1fa53ba2e, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.749209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e74745-bc2a-4d7b-8cb6-b42c34b90eb7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.760980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:43.761680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.761956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.762272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] PATH_INFO: `/v3/users/swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.765521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:43.772318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:43.773827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-95a0504c-5f0b-4f0f-a6a3-352f10dd42ff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:43.788609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:43.789753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:43.791228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 308/613] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:43.792581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a920-2f52-4399-89ad-7cf0ed13496f None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.792581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a920-2f52-4399-89ad-7cf0ed13496f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.792883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283a920-2f52-4399-89ad-7cf0ed13496f None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.795987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 309/614] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:43.798867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283a920-2f52-4399-89ad-7cf0ed13496f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:43.906553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 306/615] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:43 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/9f7790a1b25740fba9cb92c1fa53ba2e/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 502 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:38:43.907860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:43.908163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:43.916834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:43.957199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:43.958458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:43.975606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283a920-2f52-4399-89ad-7cf0ed13496f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.093813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.095424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.096264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.097081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.098912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:44.099794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.104547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.164171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 310/616] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:44.174678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.176846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 307/617] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:44.275432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.413006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484fd9a-29e6-4fee-8f82-bdf522c7df25 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.513477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.514559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.515056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.515546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.520216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.559229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:44.562037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:44.591355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:44.601124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:44.606920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:44.608593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:44.608593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:44.608593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:44.608593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 311/618] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:44.616398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.721830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9a37a-494c-4cad-9fc1-600f1c9e6593 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.772669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:44.773380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:44.773660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:44.773934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:44.789003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:44.794568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:44.796381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:44.798275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 308/619] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:44.805030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:44.938706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb644cc-96bf-485d-8f55-767dac7f3f59 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:44.939440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.950478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:44.970569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e310ff-82ce-4346-808b-a7675669a49d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.018033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.018665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.018949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.019945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.020740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.021076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.023432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.035773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 309/620] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:45.043746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.060972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.061607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.061889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.062166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.062922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.063233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.064666 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.095880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 312/621] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 480 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:45.106105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.243643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9c7fe-5390-480c-9a19-55963d09d3c4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.297392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.298468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.299003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.299609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.314779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5fd83-d99f-4a99-b8eb-a3b34adace9d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.323495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.326379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.331194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-bb116773-c350-4c63-a885-5456952ad4c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:45.355342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.357526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:45.359864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:45.359864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:45.359864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:45.359864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 310/622] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:45.360822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9b32-070c-471a-a452-b63e625cf863 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.361336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9b32-070c-471a-a452-b63e625cf863 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.361874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044e9b32-070c-471a-a452-b63e625cf863 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.371691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 311/623] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:45.371691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9b32-070c-471a-a452-b63e625cf863 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.404676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.406275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.407059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.407893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.422906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.426361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fd0396c5-7406-4d9c-b14b-481a2aeb9696 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:45.449312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:45.451081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:45.453187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:45.457824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 313/624] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 354 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:45.461517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.462691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.463671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.496738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:45.499468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:45.537508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.598999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044e9b32-070c-471a-a452-b63e625cf863 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.646235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 314/625] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:45.648515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.681035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.682145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.682665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.684407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.686191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.686736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:45.693296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:45.740461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 312/626] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:45.747694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:45.903509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58977137-f07a-4e0b-ad2c-28137eea0f7b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:45.914357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:45.993106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:45.995294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:45.996301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:45.997214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:45.998805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:45.999760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.004285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbcfe22-62bd-4438-876a-d0870aab9608 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.010877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.046054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.046699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.046971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.052203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] PATH_INFO: `/v3/users/nova` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.052454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 315/627] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.054819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.062283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.069828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:46.087814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:46.089729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:46.091669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 313/628] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:45 2022] GET /identity/v3/users/nova => generated 82 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:46.094635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.108188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.266630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6fdae-a979-42c3-8dfe-494830d1c01f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.285931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.305694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.306315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.306594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.306880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.307473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e95994-2f1d-4596-a2eb-885381e2a53c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.307628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.307891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.309387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.336288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 314/629] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.336288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.392934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552672-6659-490d-a3da-97600773b89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.394587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552672-6659-490d-a3da-97600773b89f None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.395545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552672-6659-490d-a3da-97600773b89f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.396327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28552672-6659-490d-a3da-97600773b89f None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/71620f9e7c3147829e384a5cfae55f07/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.435122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28552672-6659-490d-a3da-97600773b89f None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=71620f9e7c3147829e384a5cfae55f07, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.436643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28552672-6659-490d-a3da-97600773b89f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.480738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79e28ec-ff99-4ab2-8414-43387d8bb691 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.538717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 316/630] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:38:46 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/71620f9e7c3147829e384a5cfae55f07/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 484 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:38:46.540449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552672-6659-490d-a3da-97600773b89f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.554749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.555790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.556275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.556709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.588018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:46.601910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:46.603142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:46.604004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:46.608321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 315/631] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:46.616308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.652512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.661559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.729582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28552672-6659-490d-a3da-97600773b89f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.756113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139c7fe3-b5a1-49f2-bccc-990e0e825a8f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:46.799705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.801416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.801960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.802466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.803414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:46.803967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.805657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.817604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 316/632] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:46.828302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:46.852619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:46.853683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:46.854202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:46.854675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:46.964910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:46.966390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:46.980845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:46.988759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e82b6c-01c5-48c5-8bb5-db4b7f44ad02 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.044756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.096645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 317/633] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:47.100283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.101380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.101856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.102322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.105258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.126285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.130308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:47.181281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.182291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:47.183213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:47.183213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.183213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:47.183407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 317/634] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:47.194005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.358756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59567a6-b448-4afe-815a-962be9565e21 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.440116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.440116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.440116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.440116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.463784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.466559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-009c9afd-7597-451c-8aac-7cf128c5ea34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:47.492886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.494777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 318/635] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 390 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1aa-4d5a-45a4-aedf-4f2bffd5ce23 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1aa-4d5a-45a4-aedf-4f2bffd5ce23 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.496580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63df1aa-4d5a-45a4-aedf-4f2bffd5ce23 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.502518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 319/636] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:47.503136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b069eab5-7abf-443e-9d18-8aa686ff95ae None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.504019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63df1aa-4d5a-45a4-aedf-4f2bffd5ce23 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.548045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.548601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.548851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.549072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.549288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.549916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.550223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.551667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7bd0b5-8f03-452b-9cdc-321ae7816d3c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.569064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 318/637] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.574701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ecbd-a268-49cf-b182-1ec6c6d79aaf None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.574701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ecbd-a268-49cf-b182-1ec6c6d79aaf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.574701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2561ecbd-a268-49cf-b182-1ec6c6d79aaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.604578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2561ecbd-a268-49cf-b182-1ec6c6d79aaf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:47.606534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2561ecbd-a268-49cf-b182-1ec6c6d79aaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:47.610471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.677176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63df1aa-4d5a-45a4-aedf-4f2bffd5ce23 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.707190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 319/638] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:47.708856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a055-dbce-476f-9759-c35fba50ffa3 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.709146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a055-dbce-476f-9759-c35fba50ffa3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.709418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408a055-dbce-476f-9759-c35fba50ffa3 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.711277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 320/639] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:47.716192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a055-dbce-476f-9759-c35fba50ffa3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.731640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.732677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.733336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.733866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.734749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:47.735293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.736975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b567c50-b125-4ce0-8a33-bbc471249a14 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.755362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 320/640] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:47.757361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.757913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.758466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.781547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:47.783297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:47.869192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408a055-dbce-476f-9759-c35fba50ffa3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:47.917952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:47.919162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:47.919666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:47.920189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:47.932961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:47.934945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:47.937680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:47.944750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 321/641] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:47.946633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:47.953740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:47.954804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:38:47.955768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:47.958826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 321/642] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:47.967519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:47.972068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.241225 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14908ea-16aa-4a1d-8ded-647d23d36719 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.246437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be646f2b-1f5f-4ad1-a3db-67d1633e5679 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.296848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.297655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.298057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.298433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.314122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.317587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.317865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.318463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.318725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.318995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.319754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.320055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.321608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:48.341623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.343833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:48.346451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:48.348785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 322/644] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:48.348998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 322/644] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:48.349218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.354099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.419910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.449090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.490747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3401517-f14a-4a12-b066-d6442d34313d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.495389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2ddd34-fa19-4f93-a27f-5e86cc653047 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.541094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.542047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.542168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.542493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.542936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.543108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.543522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.543987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.562637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:48.572038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:48.574134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 323/645] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:48.576242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.654509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.656658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.750622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.762268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31005f8d-9a17-439f-ad24-bacc97ecf0e0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.782368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:48.812176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 323/646] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:48.812176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.814711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:48.815456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:48.815751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:48.816025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:48.817473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:48.818010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:48.820169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:48.847091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 324/647] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:48.847091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:48.948571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9257bc-1ddc-43ce-a122-1d1b02f77c94 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:48.966761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84458b4b-06b9-4f57-912b-7907fd484f17 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.013075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.013721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.014014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.014280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.014969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.015328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.016737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.020112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.020815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.021090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.021361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.035067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:49.036322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 325/648] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.037849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:49.046914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.048742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 324/649] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.050248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.103150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.106828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.188131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c8959-33b7-4d07-b496-08aa64f028b3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.203559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50161e4-9286-43db-8f86-2e09a6915988 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.230030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.230664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.230937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.231203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.245863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.248500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.250553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.251936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.252436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.253022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.254378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.255102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.257730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.276680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 325/650] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.276680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:49.276921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.277817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:49.278835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:49.278835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.278835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:49.278835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 326/651] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.278835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.467858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.470298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.479866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627a6adf-ade1-4d37-b2de-4c35664d860f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.484646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18247cf6-70ca-409c-91ec-b19227920472 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.545824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.546387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.546665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.546939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.549934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.551566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.552059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.552547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.553763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.554195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.556860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.564055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.566506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.571916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 327/652] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.573228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-274a9888-8f2d-4b25-a859-c0e4d88d8a54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:49.599440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.600697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:49.601833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:49.601833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.601833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:49.602004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 326/653] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.606318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d5e0-688b-4843-b2c4-d6807e58d756 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.606318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d5e0-688b-4843-b2c4-d6807e58d756 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.606663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb7d5e0-688b-4843-b2c4-d6807e58d756 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.609604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 327/654] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:49.611093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7d5e0-688b-4843-b2c4-d6807e58d756 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.738828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3569f56d-58c8-4e56-88f4-aa1435383001 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.739442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb7d5e0-688b-4843-b2c4-d6807e58d756 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.782742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.783353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.783822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.784091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.784779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.787052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.788044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.808314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 328/655] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:49.808314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.810434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.811415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.811710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.811958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.819756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.825875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.827362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:49.829460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-1ad2a7db-e662-4c61-997d-207409b7da3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:49.846664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:49.856267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 328/656] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.857453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.875894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:49.877479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:49.932135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72529c2-e579-4e3f-8918-6305201a3e78 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:49.970338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 329/657] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:49.971560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:49.988069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:49.988069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:49.988069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:49.988069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:49.992896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:49.992896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:49.992896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.004571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 329/658] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.005462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.091400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.108665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc65bc59-a8d2-41fb-9bb9-7bf0e031b258 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.124414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d4da20-e27c-4c34-9f7b-330b0c8557d8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.149512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.154532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.155903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.156389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.157114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.164273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.170628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.170628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.170628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.171093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.171279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.171279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.171550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.171653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.182634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 330/659] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.183743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:50.183862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:50.184954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:50.186204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 330/660] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:50.191236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.322671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba2daf4-a1a5-4b88-9d66-0ca0430c43e4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.338910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6fc36-5dbd-48ce-9b84-3acbada793c4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.370145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.371088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.371575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.371876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] PATH_INFO: `/v3/users/glance` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.384342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.385884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.387084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.391096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.392021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.392336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.392611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.400792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.402276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:50.410820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:50.412453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:50.413513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 331/661] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:50.414760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:50.426793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:50.428706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 08 13:38:50.431667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:50.435061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 331/662] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:50.436097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.506812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.579425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3a9178-0646-43a7-a654-5b9adff1be96 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.583534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a416d5d-e119-4ad1-9ce5-e448bda44ee9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.637275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.637945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.638229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.638563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.643926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.645420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.646150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.646829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.648548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.649639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.652617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.658506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:50.667162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:50.668941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:38:50.670977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:50.673510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 332/663] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:50.674530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.678923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 332/664] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.680979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.749032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.825974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223fd05d-9d84-41c8-853d-d9378e49901a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.839060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de2d070-a19b-486d-98ba-5d5d641e99bc None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:50.842453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:50.876378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.877646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.878287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.878898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.880442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.881099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.884009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.887035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:50.887819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:50.888178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:50.888533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:50.889391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:50.889783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:50.891369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:50.922218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 333/665] 10.222.0.97 () {56 vars in 1139 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.922218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:50.925651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 333/666] 10.222.0.97 () {56 vars in 1137 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:50.927103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.082428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d0364c-ff93-4184-a8ca-949170945de3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.109341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10eeef9-1a82-4e31-b178-c73d9b893759 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.128473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.136118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.137088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.137527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 334/667] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:51.156062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.159784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.160567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.160824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.161101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.161861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=eb8f5ef6eb4948edbf947eacffea57bb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:51.162169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.163897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.170893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 334/668] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:50 2022] GET /identity/v3/role_assignments?user.id=eb8f5ef6eb4948edbf947eacffea57bb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:51.171763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.196337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.332960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3f2d7b-cd97-4cd5-9d22-9f3ddbf2438f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.342016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b092af-0636-4c10-a27d-3ade7932999d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.395943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.396909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.397353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.397837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.397977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.398630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.399014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.399482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.477941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.528779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.530991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.542702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.545965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.598606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.706697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 335/669] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:51.708004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.779530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 335/670] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 629 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:51.781384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.812561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.841782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351baf2-67cd-4156-8304-221978ec98cb None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.883252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.884180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.884637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.885050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.897765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.899839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:51.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:51.917179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:51.918233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:51.918233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:51.918233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:51.919949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 336/671] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:51.920902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:51.929711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c63c403-e4ee-468a-9d0d-acd1f769d812 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:51.959539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:51.971856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:51.972516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:51.972884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:51.973169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:51.985609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:51.987063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3bcc7d1c-e813-40a2-8379-308b9afc8fd1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:52.004996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.007120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:52.009461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:52.009461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.009461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:52.013155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 336/672] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.014884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8677-12c9-469a-ab5e-d18ad5451bb8 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.015549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8677-12c9-469a-ab5e-d18ad5451bb8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.020924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48b8677-12c9-469a-ab5e-d18ad5451bb8 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.023096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 337/673] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:52 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:52.024221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8677-12c9-469a-ab5e-d18ad5451bb8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.052058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b3b0cb-1817-4060-b86f-1a718d51f278 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.082756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.092590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.093352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.093633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.097395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.106469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.108000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:52.133285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.134437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:52.135603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:52.135603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:52.135603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:52.135913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 337/674] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.140760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.190036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48b8677-12c9-469a-ab5e-d18ad5451bb8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.255948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.257062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.257560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.258058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.259268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:52.259810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.262262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c905d4da-22d2-452f-97d9-5e6c1b26cbee None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.281825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 338/675] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:52.283845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.284386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.284883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.294926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433533ce-f55e-4871-8ce0-d5a46ca470b0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.316892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:52.318589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:52.354419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.358140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.358761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.359040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.359314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.360005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:52.360303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.361715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.376368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 338/676] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:52.377322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.407763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 339/677] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:52.409149 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.447350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.589830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b30a4a-da99-4a3b-9364-da18c28ccfe1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.610962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93c5bd-768d-4083-a629-995bd6decd34 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.647791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.649571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.650061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.650510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.651950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:52.652790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.654700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.657352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.658075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.658803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.658803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.667908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.669500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.672181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 339/678] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:52.673303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:52.687490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.688555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:38:52.690011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:52.692416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 340/679] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.693525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.724367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.800752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:52.805714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869915b6-6e17-4743-9970-f260969ff936 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.806763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2841cb-b1aa-4803-89a1-9f25faee0065 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:52.852904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.853867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.854349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.854826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.855737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:52.856406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:52.856690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:52.856962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:52.864559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.866019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.867367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:52.868845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:52.881093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.882123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:52.883716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:52.884184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:52.885000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 341/680] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.885886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:52.892397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 340/681] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:52.892397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.001107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.012632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df93fbf-560e-44e9-aeab-ca458baef37c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.022246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42effc67-9720-4b7a-b73c-355f4d89c541 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.057940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.058675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.059017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.059509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.060241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.060637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.061997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.069641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.070816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.071228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.071815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.089615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 341/682] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.089615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.091290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:38:53.096535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:53.097633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:38:53.099360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:53.105583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 342/683] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:53.106664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.107472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.195726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8b17a2-cc27-46c2-894c-39d8179a4c32 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.230257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19664a8-07e2-4599-94e1-c2c146fa5a13 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.232582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.233200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.233485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.233759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.234458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.234767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.236246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.250802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 342/684] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.251714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.291215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.291832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.292116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.292490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.293189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.293496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.295054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.312852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 343/685] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.314482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.326541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.353769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.382807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3deb9974-ae97-4a18-8eea-fe784745fe39 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.431379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.432032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.432427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.432789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.489360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5bacb4-e84c-458d-99ad-8dc723ad9364 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.548749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.549068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.549669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.550015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.550300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.551202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.551379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.551684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.553045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.565953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 344/686] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.566921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.651314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.652981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:53.679271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 343/687] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:53.680648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.693559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0a772d-f2f3-488d-8911-711fd51803ef None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.735109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.735905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.736415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.736881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.737899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=9f7790a1b25740fba9cb92c1fa53ba2e) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.738331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.740229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68674700-1ba1-40e7-bc74-bbee654a9fd6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.747629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 345/688] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/role_assignments?user.id=9f7790a1b25740fba9cb92c1fa53ba2e&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.749063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36e06-982b-4bab-bb3b-73e5a2e0e0a0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.749426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36e06-982b-4bab-bb3b-73e5a2e0e0a0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.749706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc36e06-982b-4bab-bb3b-73e5a2e0e0a0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.751101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 346/689] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:53.756828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36e06-982b-4bab-bb3b-73e5a2e0e0a0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.826192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61e978e-c61b-400d-b2db-c95473c0d2e3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.870422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:53.871046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:53.871328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:53.871764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:53.872406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=f87892da36634672bac2c4c12a52f15e) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:53.872722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:53.874210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:53.881984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 344/690] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/role_assignments?user.id=f87892da36634672bac2c4c12a52f15e&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:53.882878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:53.953601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc36e06-982b-4bab-bb3b-73e5a2e0e0a0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:53.960388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.016600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.026387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.027563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.028540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.029123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.047690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.049316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.065819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acb0300-a475-427f-b714-f4a7a49d01b6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-cd0210b5-af56-4d71-930f-16ffedf57b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:54.070468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.071898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:54.073301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:54.073301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.073301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:54.073301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 347/691] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.078173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.078488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.078821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.105143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:54.106862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:54.123359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.123359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.123359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.123359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.142784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.142784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:54.169936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.171098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:54.172031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:54.172031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:54.172031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:54.172677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 345/692] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.173824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.236068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 348/693] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:54.242556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.367416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.381211 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eecdb-468c-4a40-85b8-8aec611db8ce None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.426474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cc9738-c5bf-4f9d-9dd1-0952682b6105 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.450007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.464678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.465302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.465583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.465857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.466523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:54.466830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.468271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.473866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.474479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.474750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.475016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.475664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:54.475960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.477528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.491420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 349/694] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.492299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 346/695] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:54.496546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.502336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.677863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d009441d-94fe-4abb-93e2-748716d73004 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.698026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f04a5-234d-4f67-8fb0-5d45c818d819 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:54.751762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.753221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.753741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.754772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.763940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:54.765145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:54.765667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:54.766310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] PATH_INFO: `/v3/users/nova` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:54.769358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.773218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.780638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:54.783268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:54.801861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.803661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:54.805484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:38:54.808437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:54.810289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:54.812191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 347/696] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:54.814569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.816243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 350/697] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:54.819117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:54.843474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.909448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:54.993483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab55fd7-bf53-4975-9962-d1aa834de74e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.017792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979cfcea-5412-421f-ad1c-e6f7def516f2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.043253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.044378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.044673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.045133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.046097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.047798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.061816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.062990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:55.063179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:55.064825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:55.066557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 348/698] 10.222.0.97 () {56 vars in 1129 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:55.073772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.074002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.074002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.074002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.074002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.079055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.119812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 351/699] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:38:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.121146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.213839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.240545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.251138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916aec64-f59b-4e4b-9602-7f400565b52a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.261153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b9e5e8-a497-4186-9fb0-7cbca5404100 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.305128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.306012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.306437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.306850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.307849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.308326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.310358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.315714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 352/700] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.321490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.330846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.332379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.333065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.333657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.335380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.336484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.339521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.367099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 349/701] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.368898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.454502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4aca353-688e-4224-82af-9e4bd893474d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.583901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ac3636-2dc1-4a61-9717-e345e470ec85 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:55.652852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.654019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.654603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.655183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.656601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:55.657256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.660167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.666707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.671087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:55.671773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:55.671982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:55.672310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:55.673993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 350/702] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:55.675746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.719357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:55.770530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:55.771999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:55.923919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 353/703] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 607 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 08 13:38:55.930988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:55.954746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420162dc-b938-42e1-97f5-015c0fc3bd16 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.020785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.030941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.031560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.031840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.032117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.032887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=71620f9e7c3147829e384a5cfae55f07) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.035176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.035176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.050193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 351/704] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/role_assignments?user.id=71620f9e7c3147829e384a5cfae55f07&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.079222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ecf352-3d0f-4acd-a2cf-9c7ccd479623 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.127748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.128501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.132687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.133012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.146397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.147871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:56.173616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:56.174659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:56.175572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:56.175572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:56.175572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:56.175572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 354/705] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:56.182233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.334067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.392482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1052f9dc-22dd-44b7-8d2c-8102ca32fa41 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.449271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.449821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.450104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.450381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.451072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.451380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.452863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b748b78f-fee8-4a50-a1bc-1fa23429f17a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.470690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 352/706] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:56.477239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.599217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d15ab7a-094f-4e79-a29b-e8e3bc7e37ae None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.611722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:56.639623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.640321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.640600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.640875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.650662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.652385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:38:56.667378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:56.668399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:56.669368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 355/707] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:56.670896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b8c7-e52c-4276-bc2b-28937497d8f0 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.671349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b8c7-e52c-4276-bc2b-28937497d8f0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.671786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216b8c7-e52c-4276-bc2b-28937497d8f0 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.675281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.678025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 353/708] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:38:56.684809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b97b5-31af-45dc-baec-0a2b95a9826d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.685224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b97b5-31af-45dc-baec-0a2b95a9826d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.685651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74b97b5-31af-45dc-baec-0a2b95a9826d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.727745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f74b97b5-31af-45dc-baec-0a2b95a9826d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:56.730122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f74b97b5-31af-45dc-baec-0a2b95a9826d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:56.845012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e23a9-1bf6-4a15-9491-3bd1dd31c789 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:56.846074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 354/709] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:56.847645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc4c97-dfcc-4c1f-9c63-d6cef3edd5c3 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.848731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc4c97-dfcc-4c1f-9c63-d6cef3edd5c3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.849296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc4c97-dfcc-4c1f-9c63-d6cef3edd5c3 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.851212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 355/710] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:56.856513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.857127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.857602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.881962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:56.883509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:56.927976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:56.928637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:56.928912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:56.929211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:56.934193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:56.934494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:56.936990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:56.965080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 356/711] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:38:56.972919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:56.999266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.043893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 356/712] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:57.044577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.154536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3136f03b-a9b3-4ef6-a153-3a580c75d7de None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.189847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.230069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.231633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.232485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.233251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.234928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:57.235865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.239512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.248464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 357/713] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:38:56 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.340757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.414370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f01480f-a6b7-45be-a983-6a35df3489af None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.469916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.470568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.470846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.471390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.485341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.486832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:38:57.508312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:57.509416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:38:57.510339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:38:57.510339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:38:57.510339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:38:57.510339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 357/714] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:38:57.516474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.580852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.582763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:57.604714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3fa06-4dfc-4e35-817a-d6a2361e3307 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.700430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f43437-8e9f-4e7a-99bb-a99d487a6564 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:57.701950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.702591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.702869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.703141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] PATH_INFO: `/v3/services/network` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.703763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.705632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:38:57.731390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:38:57.732523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:38:57.733357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:38:57.737258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 358/715] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/services/network => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:38:57.746810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.749172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:57.750074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:57.750562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:57.751122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:57.751980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:57.752518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:57.754116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:57.769760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 358/716] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:57.778988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:57.968983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406131df-02b3-4966-8f34-6830e473b8bf None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.042344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.043437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.048176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.048176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.048176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] RBAC: Adding query filter params (name=network) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.048176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.049298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.055353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.060644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 359/717] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:57 2022] GET /identity/v3/services?name=network => generated 124 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:58.070231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:58.071764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e93562-583b-43d8-bc5c-075692d37661 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.123097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:58.174740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.175496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.175824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.176875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.176875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.178472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e309ce-fb82-4290-9f89-51e86883b74a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.290099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6731e075-459f-4ddf-b441-2526ef76f7f2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:58.300965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 359/718] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:38:57 2022] POST /identity/v3/users => generated 254 bytes in 522 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:38:58.363469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:58.364617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:58.365102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:58.365559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:58.366669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] RBAC: Adding query filter params (type=network) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:38:58.366986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:58.368905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04a139-276c-4e9f-beed-8d49a55c10d5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:58.378286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 360/719] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:38:58 2022] GET /identity/v3/services?type=network => generated 124 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:38:59.076551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ca40-c45e-4cf0-bdf7-7d4efc83d5cc None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.077103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ca40-c45e-4cf0-bdf7-7d4efc83d5cc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.077103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4737ca40-c45e-4cf0-bdf7-7d4efc83d5cc None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.078693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 360/720] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:59.085434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e73da-a4c3-4dda-89a0-bdc215b91eb5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.085952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e73da-a4c3-4dda-89a0-bdc215b91eb5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.086031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e73da-a4c3-4dda-89a0-bdc215b91eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.106671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-049e73da-a4c3-4dda-89a0-bdc215b91eb5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:59.108038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-049e73da-a4c3-4dda-89a0-bdc215b91eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:59.220198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 361/721] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:38:59.394208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac4ca-0d4c-4cd7-a444-d6d9ae62da8d None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.394821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac4ca-0d4c-4cd7-a444-d6d9ae62da8d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.395332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218ac4ca-0d4c-4cd7-a444-d6d9ae62da8d None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.397260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 361/722] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:38:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:38:59.403153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.403781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.404314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.438112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:38:59.439715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:38:59.458174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:38:59.545593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 362/723] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:38:59.567654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ac4ca-0d4c-4cd7-a444-d6d9ae62da8d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.694094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218ac4ca-0d4c-4cd7-a444-d6d9ae62da8d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:38:59.734323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:38:59.734934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:38:59.735221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:38:59.735541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:38:59.736242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:38:59.737701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:38:59.895961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:38:59.919488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 362/724] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:38:59 2022] POST /identity/v3/services => generated 251 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:00.055929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb832a1-e28d-4bbd-8c6c-f12bb64b46c1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.124470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.125132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.125399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.125673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.136124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.143761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-89f6be63-dae9-48e7-b5bb-50fd2a21cfae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:00.148960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:00.158554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:00.159476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 363/725] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:00.160278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.208248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.389046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326ed128-10a2-45a2-9f49-4226ddc7a88e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.437511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0eaee-99b5-43be-96db-54a2eb935387 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.438019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0eaee-99b5-43be-96db-54a2eb935387 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.438519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d0eaee-99b5-43be-96db-54a2eb935387 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.443245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 364/726] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:00.453155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.453660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.454148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.455190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.456029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.456537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.457051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.457986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:00.458483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.460351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.471070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 363/727] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:00.478662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.491758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:00.493896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:00.607961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:00.637805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 365/728] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:00 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 08 13:39:00.674371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867734a5-e47d-4ee4-84c0-4c4cd9d1fc8d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:00.749627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:00.750380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:00.750651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:00.750926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:00.769987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:00.773416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:00.812880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:00.814777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:00.816739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:00.816739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:00.816739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:00.824708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 364/729] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:00.835524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:00.965410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b589199-3b9d-4f13-bddc-619281e44a8b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.028909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.045796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.047347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.048126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.048969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.050550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:01.051667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.054674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.088361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 366/730] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.094553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.126230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.349846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e17e6c-9633-49d0-b941-3d0572b925fb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.383548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e7f43f-8518-402c-ae63-b3e4434a9cc8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.396052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.396841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.397194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.397539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.421599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:01.425824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e7dbd5ab-31e1-472e-9d54-e72e2ad975b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:01.436169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:01.437239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:01.438091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:01.438736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.438736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.438736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.438736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] PATH_INFO: `/v3/users/swift` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.439027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 367/731] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:01.439279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df52b-4398-4ab8-a22c-23e552bf1f43 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.444438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df52b-4398-4ab8-a22c-23e552bf1f43 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.444438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07df52b-4398-4ab8-a22c-23e552bf1f43 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.447415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.448478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 368/732] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:01.449065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.449426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07df52b-4398-4ab8-a22c-23e552bf1f43 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a483999e-96f6-4543-be48-f8ba833be7bf None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:01.465625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:01.467077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:01.468244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 365/733] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/users/swift => generated 83 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:01.469009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.469009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.473787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.495536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:01.496807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:01.516191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.576851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07df52b-4398-4ab8-a22c-23e552bf1f43 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.577040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 366/734] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:01.577775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.652046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.652889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.653173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.653447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.654151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:01.654453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.656014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.672106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 369/735] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.678085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.697920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edf2839-a9eb-4da5-8694-303e87558f4f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.718914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.741137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.741753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.742037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.742315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.743116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:01.743595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.745201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.772423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 367/736] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:01.778503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:01.860739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acad72dd-8deb-495f-a289-ab7bd9bdae13 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:01.877449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:01.905527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:01.906183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.906466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.906749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.923655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:01.925157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-9a020799-aed7-4961-9d86-e63d71a8e7ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:01.943485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:01.944674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:01.945615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:01.945615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:01.945615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:01.945615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 370/737] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:01.948196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bc9-d4a8-4067-866e-ac620efe35e8 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.948196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bc9-d4a8-4067-866e-ac620efe35e8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.948196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf7bc9-d4a8-4067-866e-ac620efe35e8 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.949460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 371/738] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:01.955026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:01.955307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:01.955583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:01.976791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:01.978013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:02.054758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b79f2c-6927-4426-a5b6-773240243852 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.056798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 372/739] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:01 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 08 13:39:02.057748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.131746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.132716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.133213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.133702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.186018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.196305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df9bf1-82af-4714-ae8a-ae0b3baaba59 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.219640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.252288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.253021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.253697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.254001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.254738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:02.255062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.256592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.264097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.268458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.279021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 373/740] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:02.280032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.465817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 368/741] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:01 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 688 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 08 13:39:02.466707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.491627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db1e7d-949c-4c69-9732-b78396b9eabf None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.553873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.555343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.556271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.556718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.557826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.560358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.589522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.620437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddbf370-24a6-49ff-8a68-c20aace5345d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.664603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 374/742] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:02 2022] POST /identity/v3/services => generated 240 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:02.665513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.689717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.691975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.692558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.693464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.698145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:02.718459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:02.722783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:02.756917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:02.758183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:02.759209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:02.759791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 369/743] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:02.761657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:02.870303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7406281-bc1d-4e91-8011-9abeed491989 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.941737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:02.942966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:02.943304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:02.943618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:02.964022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:02.966494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bd4178-4097-40cf-898c-f665490afad1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:02.969986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:02.970915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:02.971774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:02.975145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 375/744] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:02.976364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.008070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.016485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.024086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.024086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.024086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.034872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.037521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:03.062228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.063244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:03.064118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:03.064118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.064118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:03.064118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 370/745] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.069892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.084229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.119410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa5490d-b1ea-4103-94f5-b376070ef4d4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.160880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.161547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.161874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.162152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.162907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.163333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.165191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.193887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 376/746] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.193887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.215357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b6be9-fc22-42ea-a769-91c75f8805b8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.275301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.276375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.276857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.277325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.278679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.279201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.281422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21284f50-5ed5-45f9-a97a-14414b005d25 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.304238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 371/747] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.304238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fcb7-1dbd-46ab-9afd-558b49b07ee5 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.304238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fcb7-1dbd-46ab-9afd-558b49b07ee5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.304238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b149fcb7-1dbd-46ab-9afd-558b49b07ee5 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.308315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 372/748] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:03 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:03.309262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149fcb7-1dbd-46ab-9afd-558b49b07ee5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.371053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.392330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b4ce0b-2926-4384-a065-09b0713f7b10 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.446710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.456030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.457105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.457592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.458075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.459338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.459875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.461463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b149fcb7-1dbd-46ab-9afd-558b49b07ee5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.462909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.486213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 377/749] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:03.488162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.522975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.524035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.524552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.525037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.545695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.548274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-b6abe870-11ec-4263-a5b2-22361e4a4755 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:03.579038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.580208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:03.581311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:03.581311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.581311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:03.581311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 373/750] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.590507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e9682-a814-4db8-a635-ff508b318463 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.590942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e9682-a814-4db8-a635-ff508b318463 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.592012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e9682-a814-4db8-a635-ff508b318463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.624590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-774e9682-a814-4db8-a635-ff508b318463 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:03.626737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-774e9682-a814-4db8-a635-ff508b318463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:03.642447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a314e6-93be-4778-aa68-e01fdb6ca0a8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.709146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.710358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.710863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.711358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.735354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.738397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.750452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 374/751] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:03 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:03.751402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e9682-a814-4db8-a635-ff508b318463 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:03.759789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.761031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:03.761916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:03.761916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:03.761916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:03.764821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 378/752] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.766527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.788489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.832903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:03.889362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e9682-a814-4db8-a635-ff508b318463 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.922333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3650e41-0c27-43b5-a0a9-dd48c24c5692 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:03.935089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.936737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.937346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.938123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.949679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.952005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:03.967785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:03.968823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:03.969706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:03.971962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 375/753] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:03.973045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:03.983314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:03.984255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:03.984789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:03.985441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:03.986453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:03.986833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:03.988926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.014218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 379/754] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.014218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.110207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.137111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a6b34-b590-44d6-a9f6-d843f99f56c6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.174970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7d8603-52c2-4c55-8cde-ea56dd14f95f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.181160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.188309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.188956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.189230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.189463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.190201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.190491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.191942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.205999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 376/755] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.206706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.248851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.249943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.250457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.250955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.252319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.252881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.255667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.277910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 380/756] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.279515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.371374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfff078e-b722-420c-be8c-49c18760bad9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.428385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.429778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.430470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.431006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.440512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9165eafb-425f-4e52-9b35-7288657ba6bd None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.443356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.445728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:04.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:04.467837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:04.469683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:04.472731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 377/757] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:04.475450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.493179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.506937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.510725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.510725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.510725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.510725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.510725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:04.522739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:04.525083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:04.526824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:04.529606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 381/758] 10.222.0.97 () {56 vars in 1129 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:04.530499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.582960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.651221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f7004-0fb2-438f-94c3-ed85bd512951 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.691532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b4ed49-69f7-43d6-bda5-b6ddbce71035 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.710347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.711446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.712228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.712714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.728646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.731502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.748441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:04.749044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:04.749327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:04.749599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:04.750337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=71620f9e7c3147829e384a5cfae55f07) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:04.750648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:04.752107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:04.756259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:04.757265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:04.758345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:04.758883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 378/759] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:04.760118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.769638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 382/760] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/role_assignments?user.id=71620f9e7c3147829e384a5cfae55f07&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:04.771903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:04.898539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.975961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:04.987434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4985b900-9330-4bb5-9ca9-d364c4eb60cf None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:04.997035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb0ff4a-c13e-496d-a809-889b896e8cb3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.070582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.071861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.072591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.073171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.074697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.075221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.076994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.078146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.078326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2bac31-f75e-4451-811c-ad0c1f117a5c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.078860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.079446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.080752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.081738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.085116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.091366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 383/761] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:05.091924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.104128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 379/762] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:05.106270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1a13-90df-44a7-8227-3b38a2283a7f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.106763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1a13-90df-44a7-8227-3b38a2283a7f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.107259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca1a13-90df-44a7-8227-3b38a2283a7f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.109753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 380/763] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:05.111094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca1a13-90df-44a7-8227-3b38a2283a7f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.277092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca1a13-90df-44a7-8227-3b38a2283a7f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.295840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df1111-fa21-4ebc-ba4e-e7500584adb0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.325555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.337368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.338171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.339840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.339840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.339840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.340521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.341860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.356870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 381/764] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:05.356870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.357560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.358545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.359455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.359951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.361387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.361954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.365289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019f2dbd-9399-4286-8856-5d8b466c3c00 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.392048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 384/765] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:05.393455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.393816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.394104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.414000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:05.415226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:05.431461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.504267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 385/766] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:05 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:05.504267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.571503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920bdcdb-ea4f-4ada-9af2-b308d7d73667 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.635195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.636634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.637375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.638020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.641611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=46beefbd50f34b43b82977125f31ffeb, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.644806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.683895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c50137-e457-44bd-b860-d398dca2cb19 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:05.730908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:05.732453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:05.733232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:05.733926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:05.735864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] RBAC: Adding query filter params (interface=public, service_id=52495a705fb04390bb5d6dd15626344c, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:05.736719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:05.739623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:05.740580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 382/767] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:05 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:05.741625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.761346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 386/768] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/endpoints?service_id=52495a705fb04390bb5d6dd15626344c&interface=public®ion_id=RegionOne => generated 194 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:05.769450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:05.798281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:05.903808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.001514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e6c7c-a2ec-4b4f-9313-48bb91de4333 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.020863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d92b2-afba-4e38-bcf9-5d2defc1f94d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.049658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.050283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.050562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.050843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.087892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.088611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.089092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.089520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.090158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.091814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.099696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 387/769] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/services => generated 772 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:06.104760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:06.195092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.197011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.273874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.313011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973336c-7391-43a4-89e2-ffdbd708d138 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.356229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.372474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.373463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.373971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.374581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] PATH_INFO: `/v3/services/network` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.375472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.378018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:06.391593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:06.392906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:06.393772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:06.397484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 388/770] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:06 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:06.406709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:06.438696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 383/771] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:05 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 697 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:06.449188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:06.629694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7840ac9-1ef5-477b-a543-41922824afb0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.659349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e598ffcd-2464-4003-bc31-7183fc1cd9ec None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.705505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.706066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.706345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.706617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.722291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.724358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.733655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.734712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.735197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.735707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.736899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] RBAC: Adding query filter params (name=network) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:06.737432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.740173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:06.741793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:06.743165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:06.744164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:06.744164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:06.744164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:06.744164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 384/772] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:06.757263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:06.757826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.760360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:06.768567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 389/773] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:06 2022] GET /identity/v3/services?name=network => generated 124 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:06.772445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:06.923691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99451be0-1ca1-40f5-82f8-049e6d5c5453 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:06.974216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:06.974751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:06.975028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:06.975303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:06.975964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:06.976309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:06.977723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:06.994743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8577346-d421-4f9f-8e24-c8e842f85ee2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.008702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 385/774] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.017464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.063215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.064531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.065045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.065536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.066748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] RBAC: Adding query filter params (type=network) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.067288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.070018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.080480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 390/775] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:06 2022] GET /identity/v3/services?type=network => generated 350 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.091024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.162246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.298057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25578e-ce2b-463d-9af9-22b2ed29a330 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.300542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.426643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.426643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.426643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.426643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.455863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.463016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.473644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307b336e-405d-4e0b-a330-26cca4e2fef4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:07.504887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:07.507401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:07.509572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:07.516811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 386/776] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 500 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:07.528378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.619917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.621469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.622524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.624672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.626304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] RBAC: Adding query filter params (interface=public, service_id=3cc4d373208c4f79ab3bfa42f6d29553, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.627474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.634139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.654850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 391/777] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/endpoints?service_id=3cc4d373208c4f79ab3bfa42f6d29553&interface=public®ion_id=RegionOne => generated 194 bytes in 564 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:07.664938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.798691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e24a2b-8c76-4816-b0cb-77e86dbcf7c6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.869362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:07.911109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:07.912225 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:07.912550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:07.912822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:07.913545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:07.913873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:07.915296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:07.929363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 387/778] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 403 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:07.930130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f9a2e4-a8f1-4714-9e64-a2f962e49be8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:07.936066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:07.995357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.013129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.014644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.015511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.016275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.018801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.021976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9454d943-80f7-494b-914b-78b95672cf0e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.038049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 392/779] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/services => generated 772 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:08.184299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20635868-6c4b-497a-9726-4ba614e40bc2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:08.267744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:08.269225 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.269981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.270675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.272250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=69ef2e0efb2143bab3ea9f3e12177331) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:08.273081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:08.275973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34de8de4-05b8-4b3b-a9b6-2eb5aac6e399 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:08.291714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 388/780] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:07 2022] GET /identity/v3/role_assignments?user.id=69ef2e0efb2143bab3ea9f3e12177331&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:08.516436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54864-9e3c-4275-91a4-ad7dba2e7f48 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.517510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54864-9e3c-4275-91a4-ad7dba2e7f48 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.518002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd54864-9e3c-4275-91a4-ad7dba2e7f48 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.520052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 393/781] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:08.528022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ae78d-c328-4433-a5a5-99af84ae68a7 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.528424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ae78d-c328-4433-a5a5-99af84ae68a7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.528627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ae78d-c328-4433-a5a5-99af84ae68a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.559103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-fa2ae78d-c328-4433-a5a5-99af84ae68a7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:08.560851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fa2ae78d-c328-4433-a5a5-99af84ae68a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:08.635223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:08.667902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 389/782] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:08.873681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d2df-f235-41af-a796-50c898fc03f4 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.874052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d2df-f235-41af-a796-50c898fc03f4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.874439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4d2df-f235-41af-a796-50c898fc03f4 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.876444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 394/783] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:08.894324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb074b-0341-49cd-b38b-f432681081f2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:08.894855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb074b-0341-49cd-b38b-f432681081f2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:08.895092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aeb074b-0341-49cd-b38b-f432681081f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:08.919157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3aeb074b-0341-49cd-b38b-f432681081f2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:08.920418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3aeb074b-0341-49cd-b38b-f432681081f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:09.022120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 390/784] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:08 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:09.035595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4d2df-f235-41af-a796-50c898fc03f4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.125163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72659-b5dd-4fc3-9c11-e3367fefc993 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.125345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72659-b5dd-4fc3-9c11-e3367fefc993 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.125618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df72659-b5dd-4fc3-9c11-e3367fefc993 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.127475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 391/785] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:09.133914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.134228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.134416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.155949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:09.157238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:09.176040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4d2df-f235-41af-a796-50c898fc03f4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.221848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.223020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.225139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.225676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.258826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 392/786] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:09.266455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:09.272532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:09.273609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:09.274457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:09.277326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 395/787] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:09.283561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.334240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.437066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd7cecb-3900-4de8-a180-c33e6514f422 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.478124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.479137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.479622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.480120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.481034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.481557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.483168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.493891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 393/788] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:09.499858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.568729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.622839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081fe60-24b9-40a8-b72d-42502b098304 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.661435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.662662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.662941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.663219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.676242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.676615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.678902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:09.695660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:09.696721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:09.697692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:09.697692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:09.697692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:09.697692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 396/789] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:09.702922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.825093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4fbba1-ad90-4547-98c5-bb333584b3cd None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.828534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.829559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:09.834294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7789b8f4-2c7c-4a71-97c3-2f2fc2a1202f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:09.867701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.868432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.868712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.868987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.869647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:09.869950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:09.871333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:09.878396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:09.879425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:09.879922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:09.880960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:09.890027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 397/790] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:09.891704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:09.897051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:09.902980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:09.903977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:09.904850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:09.907949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 394/791] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:09.909300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.001080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35ec17-512d-4583-8a86-40955503e260 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.032352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62059551-72fa-4e92-ab99-4bc11129050b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.041622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.042350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.042647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.042925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.043528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.044985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.054878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.055979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:10.056983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 398/792] 10.222.0.97 () {56 vars in 1129 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:10.057770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.083069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.085132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.085653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.086135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] PATH_INFO: `/v3/users/swift` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.091931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.097285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.099863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:10.113636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.114695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:10.115836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:10.118936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 395/793] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:10.120206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.154141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.217867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f500d3de-aa9f-4c0c-acf1-3775fe805550 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.261861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3801593-3904-48c4-843d-e2d28b848a16 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.305629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.306268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.306627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.306795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.307503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.307810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.309353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.320105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 399/794] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.321040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.338813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.339605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.339921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.340242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.340917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.341228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.342640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.348520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 396/795] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.349447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.456069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.482097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca86542d-b572-4f84-9689-c3ff3a704829 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.527916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.528483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.528777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.529025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.542600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.544106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:10.560548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.562463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:10.565724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:10.565724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:10.565724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:10.565724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 397/796] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:10.565724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.573598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.601464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80730cae-1c4b-433f-bb52-fee62d401d5c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.673795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.675362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.676517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.677335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.679366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.680348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.682654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375e9679-0d9d-454a-be9e-d78eaffb74ad None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.683808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.721158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.721774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.722055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.722318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.722968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:10.723270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.725167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 400/797] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.727077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.730039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.732032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:10.737495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 398/798] 10.222.0.97 () {56 vars in 1124 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:10.737757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.856600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0671f903-e75c-4aa7-9f15-cff566a9b041 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:10.897232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:10.898233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:10.898233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:10.898520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:10.911828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:10.913350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:10.935188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:10.937250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:10.939518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:10.939518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:10.939518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:10.939518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 399/799] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:10.939518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:10.948299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877a74c-47bb-46bd-98af-fc4cd1b59c34 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.008121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.024192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.029989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.029989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.029989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.029989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:11.029989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.032014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.059627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 401/800] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.061933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.069995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5ad2d3-2cca-43e5-bea5-b17e1eeaab49 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.083492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.111083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.111971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.112480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.112908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.113734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.115952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.279951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32cb5a7-d395-40ba-89cd-eabee5bf6a53 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.292674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 400/801] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:10 2022] POST /identity/v3/endpoints => generated 346 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:11.294772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.359643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.360949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.361513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.361924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.363175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:11.363696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.366428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526a270e-cca8-4e9e-8587-7bb96b0744b8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.396677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 402/802] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:11.396677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896748d-9413-4468-8e43-f718844d0f93 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.397075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896748d-9413-4468-8e43-f718844d0f93 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.397276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896748d-9413-4468-8e43-f718844d0f93 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.400099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 403/803] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:11.402372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f430-5914-4c9d-8c02-5e4840fed6ed None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.402865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f430-5914-4c9d-8c02-5e4840fed6ed None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.403361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a1f430-5914-4c9d-8c02-5e4840fed6ed None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.406112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 404/804] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:11.407548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1f430-5914-4c9d-8c02-5e4840fed6ed None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.510616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6b19fc-7213-493c-879c-302a998bed51 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.523415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.531980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.568924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.570294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.570839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.571361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] PATH_INFO: `/v3/users/nova` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.584190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.585172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.589284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a1f430-5914-4c9d-8c02-5e4840fed6ed None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9345723f-1b15-49d0-83e3-19b128839a8b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:11.603842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:11.605673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:11.607429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 401/805] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:11.610452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.610452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.610452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.638228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:11.640333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:11.646018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:11.647587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.648088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.648540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.658606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:11.660627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ddcbd75c-dc3c-4d2b-ae9c-3d8a91b90bc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:11.678715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:11.680912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:11.682687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:11.687296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 405/806] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:11.689104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:11.689523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:11.689922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:11.711504 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:11.713126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:11.788185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 402/807] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:11.788185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.830589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 406/808] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:11.832275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:11.934449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:11.985466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46b654b-c48b-414a-9607-7767167cedc3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:11.995394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.029565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.031003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.031750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.032748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.034499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:12.035526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.039110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.070495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd89fd0-6eef-4c37-9915-1911e3ea16fe None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.104997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 403/809] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:12.111336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.194890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020874ea-171f-4042-be62-b03f521885fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.195883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020874ea-171f-4042-be62-b03f521885fb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.196409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020874ea-171f-4042-be62-b03f521885fb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.196911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020874ea-171f-4042-be62-b03f521885fb None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.198241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020874ea-171f-4042-be62-b03f521885fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:12.198801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020874ea-171f-4042-be62-b03f521885fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.201606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020874ea-171f-4042-be62-b03f521885fb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.229002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 407/810] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:12.240676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020874ea-171f-4042-be62-b03f521885fb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.322683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f2cb62-764b-416a-bfd1-5854cecc8a82 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.370666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.371604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.371893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.372281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.403760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020874ea-171f-4042-be62-b03f521885fb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.409219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.431694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.459706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.460670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.461221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.461692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/71620f9e7c3147829e384a5cfae55f07/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.465379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=71620f9e7c3147829e384a5cfae55f07, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.467232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.490696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.493639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.592732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 408/811] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:12 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/71620f9e7c3147829e384a5cfae55f07/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:12.593800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.688119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 404/812] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:12.689338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.814912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.816245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9478d3-c551-429f-a3a4-f058d1a591a7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.877157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.877428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.877764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.877997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] PATH_INFO: `/v3/services/network` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.878561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:12.880123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:12.895369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:12.897486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 08 13:39:12.899207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:12.902030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 409/813] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/services/network => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:12.903146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:12.920016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c81172-cf62-4f4e-8375-b39187c6915d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:12.955798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:12.978001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:12.979250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:12.979692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:12.980168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:12.997995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:13.005148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:13.006901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:13.007907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:13.017676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 405/814] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:13.017676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.073189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ceffc0-a791-4a12-aa3e-6037f0563017 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.142633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.144189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.144530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.145100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.159700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.161442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:13.181434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:13.183450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:13.185327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:13.185327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:13.185327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:13.187591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 410/815] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:13.188704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.192122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54129a6-c623-4750-b00c-0a5dc6240c9b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.216937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.242297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.242932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.243921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.243921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.244260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] RBAC: Adding query filter params (name=network) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.244443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.245856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.256947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 406/816] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.258051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.428722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.450598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3e381b-f463-4f01-8620-60f438d842b4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.482083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d04de4-35bf-494e-a339-e9d88913770a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.519164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.520428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.520979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.521679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.522927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.523491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.525797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.548213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 411/817] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.548213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.570494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.572658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.573416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.574587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.576094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.576900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.580814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.607523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 407/818] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.609154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.689501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.732088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164942d6-6507-44b8-9689-024ea2838c27 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.778528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.779152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.779431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.779773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.780465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] RBAC: Adding query filter params (type=network) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:13.781471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.782908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.793715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 412/819] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/services?type=network => generated 350 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:13.793715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.811993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e7692-d636-410b-950e-b5ed278bab5e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:13.812833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:13.866586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:13.867275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.867572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.867835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.881509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:13.883029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-ee66eedd-ad33-4844-b087-dc3ed2b2a3eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:13.901727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:13.903742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 408/820] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a5a6-9e68-4607-b731-0b528a1b2031 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a5a6-9e68-4607-b731-0b528a1b2031 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:13.905595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5129a5a6-9e68-4607-b731-0b528a1b2031 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:13.907323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 409/821] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:13.907717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a5a6-9e68-4607-b731-0b528a1b2031 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:13.941307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2bc0cf-00af-4f70-a5b4-5dd29ddedc19 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.016920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.017971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.018459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.018933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.032741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.035464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:14.060352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:14.062087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 413/822] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:14.064675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.176669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.188363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.197405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5129a5a6-9e68-4607-b731-0b528a1b2031 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.218711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d59e2d-4ede-4e19-ac0c-5286db9be39f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.259493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.260907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.261384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.261936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.263049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.264560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.265576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.266200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.266483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.266905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.267575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.267881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.269307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d46ca81-ecfb-4a4d-abea-0521be90c00f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.285673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 414/823] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.287111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.293828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.293828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.309740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:14.310989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:14.373512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 410/824] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:13 2022] POST /identity/v3/endpoints => generated 337 bytes in 466 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:14.375487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.481671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 415/825] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:14 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:14.483356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.611221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af966a62-1bb3-4c18-9a2e-f616bd02f773 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.669721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.688249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.689793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.691167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.692261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.694497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.695365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.698478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.717661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551cb3eb-0189-4b62-b46d-83853a71ae0a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.726421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 411/826] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:14.734660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.756975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.776910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.777696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.777981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.778128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.788916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.790411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:14.803801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:14.805472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:14.807040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:14.809922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 416/827] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:14.811560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:14.937142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5379efe-2e07-4ca1-89e3-f3d1f5d77c61 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.957147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cf502-2f44-4949-af1b-249c857fc235 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:14.970213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:14.988492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:14.989027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:14.989366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:14.989652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:14.990592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:14.990921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:14.992464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.019900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 417/828] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.021662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.039229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.041116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.041985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.042752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.044952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.045948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.049217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.063388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 412/829] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:14 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.112874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.212459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca05c2d-41eb-472b-a3a1-8b52b66824a0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.254362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.280111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.284106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.284475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.284938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.305217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.306809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.306997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bbe1f8-e00a-4bba-a97d-8ffc3d6fc3e6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:15.335905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:15.337085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:15.338485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:15.338485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:15.338485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:15.340413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 418/830] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:15.346857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.363755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.365673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.365673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.365882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.367606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.369486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-1c3b21ce-0477-4a26-907b-81323bf84805 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:15.389599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:15.390683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:15.391514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:15.392510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 413/831] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:15.394265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387e6c-b508-45b0-9ad8-b9fbd46812f0 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.394757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387e6c-b508-45b0-9ad8-b9fbd46812f0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.396073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41387e6c-b508-45b0-9ad8-b9fbd46812f0 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.399125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 414/832] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:15.400552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387e6c-b508-45b0-9ad8-b9fbd46812f0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.405440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.499723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83f2a84-9533-49d8-9d50-7262a08453ae None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.569050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.569959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.570530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.570778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.571848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.572753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.574555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f05ed-9f95-4d6d-b21b-220a4acede40 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.589813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41387e6c-b508-45b0-9ad8-b9fbd46812f0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.600785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 419/833] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.602664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.603274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.603756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.625104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:15.626407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:15.650993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.652019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.652545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.653029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.654154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.654665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.657141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.659249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.672879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 415/834] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:15.672879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.739661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 420/835] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:15 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:15.741663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.798380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:15.809273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5cc981-31de-40f2-b3ca-f75eae682ee3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.858053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.859086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.859562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.860030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.871365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.873849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:15.890696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:15.891648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:15.892613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:15.899735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 416/836] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:15.901200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:15.940353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4a645b-1190-4326-bd04-94f1aa7ba0c1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:15.985454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:15.986517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:15.987041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:15.987552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:15.988577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:15.989307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:15.991619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.004652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 421/837] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:16.029884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.034449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.107154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903f6cae-0c11-4203-9521-3d0eeb8a60bf None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.181939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.182879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.183340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.183907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.185789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:16.186343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.190623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.222611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 417/838] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:16.228812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.247647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.269604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4701a2d-130b-4750-9c03-04f4b41b2d5a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.329728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50845636-6176-4564-8776-c037a201b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.330397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845636-6176-4564-8776-c037a201b16e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.330679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845636-6176-4564-8776-c037a201b16e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.330958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50845636-6176-4564-8776-c037a201b16e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.344620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50845636-6176-4564-8776-c037a201b16e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-50845636-6176-4564-8776-c037a201b16e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:16.350526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:16.352521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:16.354268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:16.356453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 422/839] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:16.358740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50845636-6176-4564-8776-c037a201b16e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.468645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775ac0ff-c5b4-4136-b34b-0a12e3ff8645 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.483795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50845636-6176-4564-8776-c037a201b16e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.509094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.527725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.528707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.529148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.529620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.530667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:16.531179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.532793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.546594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 423/840] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:16.548413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.548413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.548901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.549458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/69ef2e0efb2143bab3ea9f3e12177331/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.550310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.564410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:16.583942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=69ef2e0efb2143bab3ea9f3e12177331, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.587665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.673339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6148dda4-11b4-4c2f-9292-6134b673e7d7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:16.718466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:16.719721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:16.720229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:16.720690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:16.734635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:16.736072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 418/841] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:16 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/69ef2e0efb2143bab3ea9f3e12177331/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:39:16.736945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:16.759578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:16.760660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:16.761607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:16.761607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:16.761607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:16.768342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 424/842] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:16.770672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:16.994473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d070f4bd-b76e-4f0e-b9e0-3929ac296d23 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.067528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.069347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.069844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.070321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.071532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:17.072081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.074469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.099350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 419/843] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:17.105463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.181661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.235076 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5162006-7ace-4a85-84d6-681299443cc7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.279830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.280766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.281224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.281662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] PATH_INFO: `/v3/users/swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.291468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.293330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:17.308381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:17.309602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 08 13:39:17.310677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:17.317629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 425/844] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:17.320029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.357798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.450848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5ee7f1-3d13-46cc-9371-9bf503b03e53 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.498716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.500070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.500562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.501006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.502068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:17.502549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.504225 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcc091b-a8a5-4fb3-8589-e014a7c07f98 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.532416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 420/845] 10.222.0.97 () {56 vars in 1135 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:17.534786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.562150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425adc5-1a8d-4187-b36e-b84f257da5a5 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.562715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425adc5-1a8d-4187-b36e-b84f257da5a5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.563135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425adc5-1a8d-4187-b36e-b84f257da5a5 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.566199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 421/846] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:17.572429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.572429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.572750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.594416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:17.596196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:17.682564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d720b68-a048-4e5c-87bb-b9e42db80e4c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:17.692186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 422/847] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:17 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:17.713267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:17.729888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:17.730669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:17.730956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:17.736930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:17.817828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:17.820314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:17.952819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 426/848] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:17.959347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:17.983493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.010793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.094178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2d21e8-db66-41cc-8359-c0f293a02755 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.096692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.136984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0b5a84-024d-4b05-ad4e-ddfe81136bf5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.140249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.141365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.141863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.142418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.156229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.158229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:18.178804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.179916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:18.180947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:18.180947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:18.180947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:18.181586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.182294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.182583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.182853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.183455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.188959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.189450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 423/849] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:18.191029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.196320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.197652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:18.198941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:18.201941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 427/850] 10.222.0.97 () {56 vars in 1132 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:18.203238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.289015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.308571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.329575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fef5c6-4911-4eb2-9a5a-fe7c210697fe None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.333390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc4537-57ab-4ce3-9c93-ac28dfcb6e46 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.378576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.379624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.380166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.380678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.381806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.382520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.384410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.398656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.399673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.400176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.400685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.402424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.402424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.406885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 424/851] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:18.407180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.408769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.416904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 428/852] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:18.422322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.544653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823704-1d44-4940-8f48-7244eca737e0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.571426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8b520a-79d6-41db-b43e-d44bbeb715d6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.578069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.586833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.587434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.587767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.588109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.596429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.600552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:18.611856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:18.613120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:18.614259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:18.620840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 425/853] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:18.625771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.629616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.631357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.631739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.632090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.632942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.633317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.634805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.644253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 429/854] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:18.774395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9804d675-83e6-4201-bd73-71dc2f700d4f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:18.821188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:18.822190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:18.822848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:18.823361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:18.824367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:18.824897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:18.826717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e3c6c-560d-460c-9ff4-06205e8e9c1c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:18.848305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 426/855] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:18.851150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:18.872109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:18.980751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c832dec-dbc0-41ee-a236-50eb1d7d33ec None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:19.029475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:19.030489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.030987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.031477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.032473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=71620f9e7c3147829e384a5cfae55f07) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:19.032998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:19.034775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961ed55-61a5-4edb-9a28-85280cd8bbfa None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:19.043087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 430/856] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:18 2022] GET /identity/v3/role_assignments?user.id=71620f9e7c3147829e384a5cfae55f07&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:19.822684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50139bc-1ca9-42ce-8230-57e05e977017 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.823220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50139bc-1ca9-42ce-8230-57e05e977017 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.823633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50139bc-1ca9-42ce-8230-57e05e977017 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.826577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 427/857] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:19.836029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc16ce2-1d15-4066-8737-242637cde81c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:19.836612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc16ce2-1d15-4066-8737-242637cde81c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:19.837088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc16ce2-1d15-4066-8737-242637cde81c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:19.869078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ffc16ce2-1d15-4066-8737-242637cde81c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:19.871126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ffc16ce2-1d15-4066-8737-242637cde81c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:20.004175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 431/858] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:19 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:20.502761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50139bc-1ca9-42ce-8230-57e05e977017 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:20.506279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44a7e-5d00-4e96-b13d-0668bb6a3c98 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.507300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44a7e-5d00-4e96-b13d-0668bb6a3c98 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.507735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c44a7e-5d00-4e96-b13d-0668bb6a3c98 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.510503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 432/859] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:20.520744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50389-e778-499f-a470-5b31981334b6 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.521241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50389-e778-499f-a470-5b31981334b6 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.521728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a50389-e778-499f-a470-5b31981334b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.545191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.555106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-70a50389-e778-499f-a470-5b31981334b6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:20.557379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-70a50389-e778-499f-a470-5b31981334b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:20.607700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:20.717645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 433/860] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:20 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:20.724698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50139bc-1ca9-42ce-8230-57e05e977017 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:20.805880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:20.806943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:20.807540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:20.808039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:20.809320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:20.812326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55445cea-3928-4fac-9478-466b005dea5d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:20.950874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 428/861] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:20 2022] POST /identity/v3/services => generated 251 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:21.103874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96936-d7bc-4525-a88c-be50a02767d8 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.104840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96936-d7bc-4525-a88c-be50a02767d8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.104840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e96936-d7bc-4525-a88c-be50a02767d8 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.106120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 434/862] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:21.113471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.113979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.114527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.149371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:21.151442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:21.224899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96936-d7bc-4525-a88c-be50a02767d8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.249112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 429/863] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:21 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:21.421268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e96936-d7bc-4525-a88c-be50a02767d8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.477132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.478166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.478732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.479303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.482954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.493785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:21.500158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:21.501390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:39:21.502341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:21.503844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 435/864] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:21.513701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.558010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.583523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.700776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36441cb-4153-43ad-833a-bba331c9d4fa None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.716065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7a407c-f7e6-43ec-815b-72acc94ebff9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.742460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.742981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.743265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.743544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.744238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:21.744537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.745989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.756074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 430/865] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:21.765716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:21.791919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.793188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.793768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.794331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.795712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.798533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.889293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:21.896815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e040dc-9005-4394-9c4b-5da1a717a374 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:21.921336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 436/866] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:21 2022] POST /identity/v3/services => generated 269 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:21.953712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:21.954680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:21.954916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:21.955141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:21.971503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:21.973502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-212b253e-4578-4d9e-9b31-9f68c6412ad4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:21.998810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:22.002297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:22.005572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:22.005572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:22.005572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:22.005572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 431/867] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:22.007435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.140050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.203525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662db5f7-87c3-4400-b79c-b73d96073325 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.247124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a8a2-b5fe-446b-96ba-85e984f92006 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.247454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a8a2-b5fe-446b-96ba-85e984f92006 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.247667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994a8a2-b5fe-446b-96ba-85e984f92006 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.249321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 432/868] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:22.254880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.255589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.255589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.273244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82189553-c763-446b-bb17-6b509b661727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.274737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189553-c763-446b-bb17-6b509b661727 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.275204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189553-c763-446b-bb17-6b509b661727 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.275635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82189553-c763-446b-bb17-6b509b661727 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.276840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82189553-c763-446b-bb17-6b509b661727 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:22.277381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82189553-c763-446b-bb17-6b509b661727 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.278413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:22.279764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:22.279884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82189553-c763-446b-bb17-6b509b661727 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.306635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 437/869] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:22.315751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82189553-c763-446b-bb17-6b509b661727 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.402921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 433/870] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:22 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:22.474991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82189553-c763-446b-bb17-6b509b661727 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.496371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.518158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.518723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.519000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.519269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.528269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.529738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:22.543729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:22.545516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:22.546439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 438/871] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:22.554389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.697344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:22.730145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca89ca8-a5d6-4e05-9baf-777fa95c10cc None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.781441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.782532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.783028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.783496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:22.784695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:22.785230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:22.787006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:22.810102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 434/872] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:22.817413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.817883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:22.950719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d343ca-dea5-415a-8908-a400023fd94b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:22.997395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:22.998397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:22.998888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:22.999360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.025722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08da4ef0-4ea9-41a2-bb33-c548c4ccd8b7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.067759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.093781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.107922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.108988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.109313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.109424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.109859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.111054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.111757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.113601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:23.132194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:23.134043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:23.135731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:23.148099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 435/873] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:23.151807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.264699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 439/874] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:22 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:23.271297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.323328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c91c4e-fa7d-45be-b40d-b532997074cd None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.369662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.370376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.370722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.371082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.371808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:23.372196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.374308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.380783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 436/875] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:23.386927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.443960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.483982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.495253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a08b0d-e752-4855-8ed1-d757ad277a32 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.546947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.548224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.549137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.549632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.561662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566c877f-fdca-47d4-9b0c-c3ad78659560 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.565012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.567858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:23.588167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:23.589431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:23.590359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:23.590359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:23.590359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:23.597323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 440/876] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:23.598445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:23.639932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.641284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.641869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.642592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.644339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:23.644788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.647825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.664180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 437/877] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:23.782207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dc346a-0e38-4a63-baaa-634568c86ba9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:23.822452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:23.831631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:23.832083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:23.832409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:23.835324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:23.835324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:23.835324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:23.835769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:23.851999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 441/878] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:23.858510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.066821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a194c312-c3b3-4734-9976-55bc057634bc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.096816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.110394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.110944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.111255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.111534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.119904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.121443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:24.134998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:24.137330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:24.139428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 438/879] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:24.142943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.276523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e084727-52cf-4b99-b210-9246e7ef0cf4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.332187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.332187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.332187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.332187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.332813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:24.332813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.332813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d047f2fb-7d58-4d9a-bfbd-010ed6432928 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.353410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:24.354318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 442/880] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:24.524631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36cdce2-82cd-45ee-a29b-a7803064d316 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:24.593232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:24.594771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.595580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.596386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.598063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=69ef2e0efb2143bab3ea9f3e12177331) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:24.598987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:24.602904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97feca-b7ea-430d-9189-ea6d4c62eaee None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:24.616681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 439/881] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:24 2022] GET /identity/v3/role_assignments?user.id=69ef2e0efb2143bab3ea9f3e12177331&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:24.634080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263118-53ed-4e0a-b0f6-769109b9dee1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.634574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263118-53ed-4e0a-b0f6-769109b9dee1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.635080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39263118-53ed-4e0a-b0f6-769109b9dee1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.638250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 443/882] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:24 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:24.648308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.648799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.649302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.673559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ed33-45e1-4d72-b54b-56856ea7bb2e None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.673853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ed33-45e1-4d72-b54b-56856ea7bb2e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.674200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9073ed33-45e1-4d72-b54b-56856ea7bb2e None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.675832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 444/883] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:24.682552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:24.682864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:24.683178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:24.684491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:24.686820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:24.703380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:24.704558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:24.764563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.792524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:24.801020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 445/884] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:24.820063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 440/885] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:24 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:25.095342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.160547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.258500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7f6a76-1fd3-4a80-ae2f-6edbd702cf62 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.297279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63edbc4-176d-4687-88a6-dc6736491ec7 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.319469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.320763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.321193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.323685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.324643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.326189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:25.351582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:25.353901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:25.355962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:25.365219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 446/886] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:25.365219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.393075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.393725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.394002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.394271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.394858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.396384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:25.419843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:25.428043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:25.430219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 441/887] 10.222.0.97 () {56 vars in 1132 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:25.432089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.464677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.520975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.536946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70321c2f-2611-41ee-8386-6e827f78f4ab None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.584213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.585433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.585716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.586134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.586638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.586941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.588579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.600815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 447/888] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.602208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.621739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104db55c-7cf1-472b-8e20-459178217aec None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.735011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4d9d5-8de6-426d-9cc2-f1ec5e5a3232 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.745370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.746427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.747086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.748750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.748898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.751562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.760502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.763689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.783358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.783996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.784393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.784667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.785319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.785654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.786727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 442/889] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 359 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.787107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.792180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.807532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 448/890] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.808018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.928087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53af7a1c-660e-40d2-bee2-be82b100b3b1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.936185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ebf5b-9d45-422f-8555-6a133553a512 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:25.945619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:25.969056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.969686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.970064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.970255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.970971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] RBAC: Adding query filter params (interface=public, service_id=92659118199a4970a29179f99c0e6ce7, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.971327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.972857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:25.984093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 449/891] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/endpoints?service_id=92659118199a4970a29179f99c0e6ce7&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:25.985055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:25.985187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:25.986022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:25.986602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:25.987078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:25.987981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:25.988595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:25.990366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.002035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 443/892] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:26.003527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.026463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.106161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842aaf9-8080-49a8-99aa-c0747fa0ceb6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.137288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4897e320-5e4d-4f56-99b0-77a7fded7c5f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.146439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.147072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.147398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.147713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.148371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.149857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.164608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 450/893] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:25 2022] GET /identity/v3/services => generated 1268 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:26.186472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.187950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.188677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.189211 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.190298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] RBAC: Adding query filter params (interface=public, service_id=344cce29448c42a4848d791ceef8da67, region_id=RegionOne) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:26.190900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.192667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b438ef-26e9-4817-ab63-898e8727187b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.203113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 444/894] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/endpoints?service_id=344cce29448c42a4848d791ceef8da67&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:26.209786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:26.350936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:26.355901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c2b5e9-66c2-49f1-a00c-a2a608ae2819 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:26.403077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:26.404106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:26.404420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:26.404694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:26.405324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:26.406980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17f418-48da-4865-884b-f721e8dfefe7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:26.414307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 451/895] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:26 2022] GET /identity/v3/services => generated 1268 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:39:27.546322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f56a-8d02-4790-8e55-27d8b7962ad8 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.547118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f56a-8d02-4790-8e55-27d8b7962ad8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.547118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362f56a-8d02-4790-8e55-27d8b7962ad8 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.548873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 445/896] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:27.560187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca44b1-f80f-452b-be99-438fc5d147ed None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.560187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca44b1-f80f-452b-be99-438fc5d147ed None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.560187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fca44b1-f80f-452b-be99-438fc5d147ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.594036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5fca44b1-f80f-452b-be99-438fc5d147ed None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:27.596716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5fca44b1-f80f-452b-be99-438fc5d147ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:27.737360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 452/897] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:27.879700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2fc8d-bbb9-4733-9db6-b2c637bc7bee None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.880174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2fc8d-bbb9-4733-9db6-b2c637bc7bee None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.880603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2fc8d-bbb9-4733-9db6-b2c637bc7bee None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.883554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 446/898] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:27.892540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:27.893008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:27.893494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:27.925044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:27.927130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:28.064338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 453/899] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:27 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:28.243537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2fc8d-bbb9-4733-9db6-b2c637bc7bee None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.313954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.445618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd2fc8d-bbb9-4733-9db6-b2c637bc7bee None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.512849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.513886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.514374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.514841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.516006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:28.518685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccca41d-e3dd-45d3-b3c1-82620434e990 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:28.536961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.570913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.632359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 447/900] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:28 2022] POST /identity/v3/services => generated 241 bytes in 387 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:28.636864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea15e-9af2-4017-9244-b804575b1f72 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.637281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea15e-9af2-4017-9244-b804575b1f72 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.637482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccea15e-9af2-4017-9244-b804575b1f72 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.639141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 448/901] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:28.645190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238fe69-2c75-41e4-a3b0-a14f76fbee50 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.645599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238fe69-2c75-41e4-a3b0-a14f76fbee50 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.645773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7238fe69-2c75-41e4-a3b0-a14f76fbee50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.668728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7238fe69-2c75-41e4-a3b0-a14f76fbee50 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:28.670034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7238fe69-2c75-41e4-a3b0-a14f76fbee50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:28.717839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2a5602-a20e-48fa-81d9-4729f1cd90ba None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:28.787098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 449/902] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:28.787354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:28.787731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.788015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.788397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.789619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caac8-466c-460a-981a-6c23c995e08a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.790112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caac8-466c-460a-981a-6c23c995e08a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.791034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0caac8-466c-460a-981a-6c23c995e08a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.794447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 450/903] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:28.801894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:28.802396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:28.803555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:28.807522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:28.813291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:28.814290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:28.815385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 454/904] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:28.822220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:28.845402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:28.849005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:28.878274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.967701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:28.969228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 451/905] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:28 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:29.023113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187211c2-1e8b-4225-96b6-d770c6550f92 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.069369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.095778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.097225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.097708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.098184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.099387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.099918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.103483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.120646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 455/906] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:29.128640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.229928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a8da38-3f0c-49a9-a0a8-af6b13e8d141 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.297234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.298280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.298738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.299197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.300271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.302905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.323108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.324585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.325528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 452/907] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:29.326910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.335702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b0c19-880e-4947-9568-02bb54719d31 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.357303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.406084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.407102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.407588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.408058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.427909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.432985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.436288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:29.465462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.467268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:29.469631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:29.469631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:29.469631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:29.469631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 456/908] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:29.469631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.514381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92714aeb-d843-4c8d-92cd-72bfaf07113a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.579081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.580059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.580587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.581074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.582131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.584540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:29.599660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:29.600828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 08 13:39:29.601821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:29.608414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 453/909] 10.222.0.97 () {56 vars in 1132 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:29.609144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e42d8-4a53-424f-a831-caf06e9b05b1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.609596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.654627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.656401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.656831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.656988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.657571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.657863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.659274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.672762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 457/910] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.672762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.746053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.772653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:29.815877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7aee9b-c4cd-42df-8589-def8e67bea3c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.816317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4495c-4e9a-49e6-af81-a822ac9bd497 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:29.863696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.864441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.864809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.865147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.865866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.866297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.868036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.874278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 458/911] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.875716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:29.896295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:29.897862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:29.898621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:29.899330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:29.901197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:29.901701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:29.904591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:29.937491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 454/912] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:29.937491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.032842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6326916-e798-4067-9c2b-e521c4d7c834 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.060652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.078697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.079317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.079721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.079991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.080738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.081041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.082427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.096232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 459/913] 10.222.0.97 () {56 vars in 1114 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.096232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.096677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc793d18-05c7-4235-bfbb-0c01892515cb None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.137720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.138248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.138534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.138809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.139450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.139753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.141176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.147492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 455/914] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.148550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.151606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.252575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d719bba7-e20d-42bb-ae55-bf5a4fb004a5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.272366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ee64-5064-45bf-96e9-12acd39dfe20 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.295956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.296724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.297062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.297434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.305899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.307328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.315532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.315532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.315532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.315532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.315985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.315985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:30.325621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:30.326625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:30.327670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 460/915] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:30.329097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.385932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.437855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 456/916] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:30 2022] POST /identity/v3/endpoints => generated 346 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:30.439030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.474073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f008378-8492-42ef-a8c6-00076415db95 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.576163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.577621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.578020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.578585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.579832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.581627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.582879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.652094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6df56c8-7131-4aec-be26-c3eaf3d3dc6b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:30.703068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:30.703973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:30.704557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:30.705024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:30.705946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:30.706459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:30.708516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:30.741291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 461/917] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:30 2022] POST /identity/v3/endpoints => generated 347 bytes in 406 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:30.747585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 457/918] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:30.757708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:30.976114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:30.998156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8d52df-0a5b-4038-a854-371de50e8c81 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.039707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.040389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.040722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.041007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.140409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.142273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14c149c-954a-48fb-8b93-21eb958eff38 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.258184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.271069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 462/919] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:30 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:31.277469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.358286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d026e-15d6-420f-96c7-2d86e350fbf0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.358625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d026e-15d6-420f-96c7-2d86e350fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.358851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23d026e-15d6-420f-96c7-2d86e350fbf0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.360622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 463/920] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:31.366524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.367368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.367641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.396479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:31.397796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:31.414979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5628687-93c9-41e9-afa3-6abb712bb2b3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.436468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.458042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.458697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.458999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.459421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.474443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.475999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:31.494137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:31.496695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:31.498629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:31.498629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:31.498629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:31.498629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 458/921] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:31.501947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.520272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 464/922] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:31 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:31.653560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212d7d42-75ad-4f07-802a-e0474a3acdc3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.722875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:31.723925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:31.724472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:31.724969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:31.726738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:31.727270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:31.729983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:31.755276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 459/923] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:31.760825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.857016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:31.915185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:31.955970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4408bf-1022-4def-927f-c9554ca6e69e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:31.990992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.008383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.009793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.010298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.011499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.021459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.025559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:32.042042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:32.043131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:32.044012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:32.044491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414b174-8580-4058-ad0f-3b80f0f9d99c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.047996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 465/924] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:32.056743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.095179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.096667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.097751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.098270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.099634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.104203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:32.116076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:32.117400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:32.118343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 460/925] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:32.123608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.188443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc35b12-2d7a-4350-a42e-afecad86d926 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.226784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.229515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.230396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.230912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.231436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.232352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.235685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.235685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.237645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c082fd-bd4f-4e20-a147-976361577a57 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.253393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 466/926] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.254613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.277091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.277845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.278392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.278392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.279178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.279549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.281168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.291824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 461/927] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:32.293518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.328616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.379584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344ba7c1-b9d4-4934-b4da-c666ef9bcab1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.412897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1e856-1175-45e3-85fb-4196d22b6fa2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.420104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.421103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.421692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.422193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.423279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] RBAC: Adding query filter params (role.id=a98ede77714744cf8742d28cbcd62a5a, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=69ef2e0efb2143bab3ea9f3e12177331) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.426816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.428800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.440706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 467/928] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/role_assignments?user.id=69ef2e0efb2143bab3ea9f3e12177331&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=a98ede77714744cf8742d28cbcd62a5a => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.458201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.458839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.459127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.459406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.460070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.460565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.462038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.474044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 462/929] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.477345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.605589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203ee90f-d8b9-4791-97e2-6408f2f180f7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.614134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.653332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.653887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.654172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.654449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.655181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] RBAC: Adding query filter params (interface=public, service_id=d17db9bf8871471982d5d7b5a6eccfe7, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:32.655490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.657117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff414a4-abcd-4d94-96a8-da7a8168f094 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.668289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 468/930] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/endpoints?service_id=d17db9bf8871471982d5d7b5a6eccfe7&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:32.670207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:32.697868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:32.792569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ed9d68-b411-47c2-bff7-9207cd9145be None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:32.834202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:32.834834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.835119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.835516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.836230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:32.837693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f1447-b37e-49b5-984a-864e4e5e2627 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:32.852329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 463/931] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:32 2022] GET /identity/v3/services => generated 1497 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:32.958534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac37-0428-4e65-8c6a-41671fc5a38a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.959037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac37-0428-4e65-8c6a-41671fc5a38a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.959037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1470ac37-0428-4e65-8c6a-41671fc5a38a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.963440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 469/932] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:32.973497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca0f7-3224-43f6-919a-c19dd78d08b1 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:32.973930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca0f7-3224-43f6-919a-c19dd78d08b1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:32.974335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9ca0f7-3224-43f6-919a-c19dd78d08b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:32.996994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8f9ca0f7-3224-43f6-919a-c19dd78d08b1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:32.998256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8f9ca0f7-3224-43f6-919a-c19dd78d08b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:33.117796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 464/933] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:32 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:33.277159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b191-74b1-4ae4-9e74-5d0f93f4b88d None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.277437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b191-74b1-4ae4-9e74-5d0f93f4b88d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.277711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32b191-74b1-4ae4-9e74-5d0f93f4b88d None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.279415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 470/934] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:33.285216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.285834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.285834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.306181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:33.307706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:33.327550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.407413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32b191-74b1-4ae4-9e74-5d0f93f4b88d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.412182 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 465/935] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:33 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:33.534420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32b191-74b1-4ae4-9e74-5d0f93f4b88d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.575006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.575972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.576477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.576960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.577850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.579493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.582600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING keystone.server.flask.common [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 08 13:39:33.647389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.sql.core [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:33.647389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:39:33.647389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [parameters: {'id': '343d9102aa7440399150b128a7344d70', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:39:33.647389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88421) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 343d9102aa7440399150b128a7344d70.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 343d9102aa7440399150b128a7344d70. Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:33.656133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 08 13:39:33.657091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application yield resource Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise self.value Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application session.commit() Sep 08 13:39:33.658161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:39:33.659048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:39:33.659972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:33.661056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [parameters: {'id': '343d9102aa7440399150b128a7344d70', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.662044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:33.662968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 343d9102aa7440399150b128a7344d70. Sep 08 13:39:33.664478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:33.666954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 471/936] 10.222.0.97 () {60 vars in 1140 bytes} [Thu Sep 8 13:39:33 2022] POST /identity/v3/projects => generated 262 bytes in 260 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:33.673251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.731567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.794851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.814894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4afb2dd-85fd-4521-aaf7-e3dbbcb8d38f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:33.864390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:33.865386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:33.865928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:33.866389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:33.874795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:33.876671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:33.889795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:33.890734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:33.892300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:33.899766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 466/937] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:33.902515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:33.906887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:33.957039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de62a2e0-57b7-44a4-8cac-4550dd475efe None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.012631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.013680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.013862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.020178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.020178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.020178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:34.028792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:34.031144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:34.033021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 472/938] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/services/compute => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:34.045234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.082134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ce944-ae46-4089-99f5-5dd33984120c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.154975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.154975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.154975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.154975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.155658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.155658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.155658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.178969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 467/939] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:33 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:34.234460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.254832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324985c3-44c2-4b53-b190-fab43c4d6448 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.328997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.330036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.330532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.331021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.332294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.332922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.336122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33862a33-5eda-4f78-9d5f-3ed4e49a9700 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.354779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 473/940] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:34.363291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:34.469763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:34.523135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dc9f9e-ffa3-46fc-9efa-ff45696f5840 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:34.571376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:34.571908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:34.572237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:34.572499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:34.573153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] RBAC: Adding query filter params (type=compute) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:34.573456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:34.574856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb5ebc8-7413-45f5-bc80-ad55d00e5d2d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:34.581089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 468/941] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:34 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:35.009352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23cf4c-4b8d-4dfd-b630-17d23ac7739f None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.009615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23cf4c-4b8d-4dfd-b630-17d23ac7739f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.010005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf23cf4c-4b8d-4dfd-b630-17d23ac7739f None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.011625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 474/942] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:35.017408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d93f6-63ff-4c6b-bc6a-0495857aca8d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.017697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d93f6-63ff-4c6b-bc6a-0495857aca8d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.017997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1d93f6-63ff-4c6b-bc6a-0495857aca8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.039116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-da1d93f6-63ff-4c6b-bc6a-0495857aca8d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:35.040401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-da1d93f6-63ff-4c6b-bc6a-0495857aca8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:35.173660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 469/943] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:35.369515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048e3a-ff61-4f2a-87ad-40939f790c5c None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.370480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048e3a-ff61-4f2a-87ad-40939f790c5c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.370480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f048e3a-ff61-4f2a-87ad-40939f790c5c None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.371735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 475/944] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:35.378476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.379367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.379367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.400573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:35.401804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:35.488242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.514615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 470/945] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:35 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:35.525008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048e3a-ff61-4f2a-87ad-40939f790c5c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.738271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f048e3a-ff61-4f2a-87ad-40939f790c5c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:35.806984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.810695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:35.811697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:35.812232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:35.812798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:35.826464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:35.832397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:35.833447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:35.834393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:35.838247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 476/946] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:35.844284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:35.869164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:35.988764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1387068-a630-4438-8ec4-b7bbe3f0dafb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.031746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e0c8ab-805d-4c4a-8802-4918a4bd7d2b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.057048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.057681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.057964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.058243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.058943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.059248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.060833 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.076523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 477/947] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.077613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.105933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.107032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.107494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.107993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.109256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.112046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.127793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.128887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:36.129740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 471/948] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:35 2022] GET /identity/v3/services/object-store => generated 93 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:36.137639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.219260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.230231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.276070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c4573-1a3f-42cb-88e6-429499f6ff3d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.286456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabbcb4d-f003-4cea-a34f-271c9a161977 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.339377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.340296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.340903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.341256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.342268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.342583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.344010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.350849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 472/949] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.355680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.356430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.357623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.358195 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.358747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.383929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.387256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:36.420082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:36.421329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:36.422387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:36.422387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:36.422387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:36.428926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 478/950] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:36.432583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.511008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67bd9-c01f-4250-a5d9-4cc4f9664d0f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.572001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.572923 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.573359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.573811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.574627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.575087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.576882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.584769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.588572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 473/951] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:36.590736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.649612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae321d-23b7-4731-8a3b-a5281f8fc76a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.704465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:36.719626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.720384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.720674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.720950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.721631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:36.721962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.723353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.738491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 479/952] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:36.747331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:36.769528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586a1dc-ef51-4393-988b-760ec42747cc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:36.852536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:36.853932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:36.854754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:36.855432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:36.856924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:36.860641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243f1582-776b-48f2-a901-98ce7a18d120 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:36.964957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04dfaef8-812d-463b-8c65-f19a919b3364 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.020713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 474/953] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:36 2022] POST /identity/v3/endpoints => generated 349 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:37.023697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286fb4-7a40-418b-bac6-8778e57765f3 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.024227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286fb4-7a40-418b-bac6-8778e57765f3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.024703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df286fb4-7a40-418b-bac6-8778e57765f3 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.031821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 475/954] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:37.035731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.036264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.036749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.055539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.056671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.057112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.057562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.080200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.080485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.091291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:37.093565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:37.109294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.110447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:37.111475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:37.112122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 480/955] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.121235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.170449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.234591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.263534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 476/956] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:37.345546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b7575e-c9b8-4cf7-985e-6caa120fd9bc None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.405895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.408269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.409092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.409845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.411520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:37.412357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.415839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.460116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 481/957] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:37.466295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.671908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.688898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3590a8a0-a44c-485a-ad37-7696372e5a62 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.757183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.758846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.759907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.760445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.780585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.783439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-b569bc30-ef22-49c9-b6ea-7d6ebd3cbf6d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:37.813613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:37.814650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:37.815551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:37.815551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:37.815551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:37.816271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 477/958] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:37.818574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3af-00a0-4b70-86d3-e49180e8b9f4 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.821206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3af-00a0-4b70-86d3-e49180e8b9f4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.821206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebea3af-00a0-4b70-86d3-e49180e8b9f4 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.822833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 478/959] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:37.824382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebea3af-00a0-4b70-86d3-e49180e8b9f4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:37.866247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.868101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a60e550-ba73-48e8-8cb5-df70e09d7eb5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:37.871288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:37.926422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6953772a-3504-4166-8008-f6073125629e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:37.927304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953772a-3504-4166-8008-f6073125629e None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:37.927756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953772a-3504-4166-8008-f6073125629e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:37.928300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6953772a-3504-4166-8008-f6073125629e None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:37.929950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953772a-3504-4166-8008-f6073125629e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:37.932951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6953772a-3504-4166-8008-f6073125629e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:37.991603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.sql.core [None req-6953772a-3504-4166-8008-f6073125629e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:37.991603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:39:37.991603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [parameters: {'user_id': 'cc06b343e3e8461db2ca731653da1fa6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:39:37.991603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=88421) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-6953772a-3504-4166-8008-f6073125629e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:38.005162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:38.006480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application yield resource Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application next(self.gen) Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.rollback() Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.force_reraise() Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise self.value Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application session.commit() Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 08 13:39:38.008133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._prepare_impl() Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.session.flush() Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._flush(objects) Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application compat.raise_( Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application flush_context.execute() Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rec.execute(self) Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 08 13:39:38.009337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 08 13:39:38.010409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result = self._query(query) Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application conn.query(q) Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application result.read() Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 08 13:39:38.011357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cc06b343e3e8461db2ca731653da1fa6', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.013737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 08 13:39:38.015353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.016968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 482/960] 10.222.0.97 () {60 vars in 1131 bytes} [Thu Sep 8 13:39:37 2022] POST /identity/v3/users => generated 160 bytes in 345 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.019338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.019844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.020369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.023860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebea3af-00a0-4b70-86d3-e49180e8b9f4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.053072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:38.057002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:38.069317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.070874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.071363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.071850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.072907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.073528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.075036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.096099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 479/961] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.096099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.199283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.204336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 483/962] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:38.205454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.241008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165a63cf-a126-4af7-8ccb-1af334dccda6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.307356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.308034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.308342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.308618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.323611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.325148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.330742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.342643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47037976-eb42-4d0d-802f-951e62943dd2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:38.344555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.345632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 08 13:39:38.346591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:38.349556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 480/963] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users/cinder => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.356121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.394157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.395703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.396494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.397142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.407694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.410458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:38.427098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:38.428108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:38.429038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:38.431357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 484/964] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:38.440530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.481245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07ef748-8e89-4ef7-88c6-e33255ae1810 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.521586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.522365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.522746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.523120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.524534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.525053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.526650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.564589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.600871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 481/965] 10.222.0.97 () {56 vars in 1101 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.602947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.624180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d42b5a6-5993-4c59-bc4e-56864d5972ba None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.675531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.676389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.676771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.677181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.678050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:38.678513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.681791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.712712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 485/966] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:38.721162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:38.784677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:38.790539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec47be9d-b2c5-4e4e-b385-5e51ab1fd0e5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:38.842289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:38.842924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:38.843205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:38.843480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:38.844300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:38.846451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8159c893-f337-432b-8173-d6aa8df96c4f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:38.929367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 482/967] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:38 2022] POST /identity/v3/services => generated 242 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:38.940818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d0df39-760b-4fc6-a220-a67b5e81161e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:39.010236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:39.011410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.012044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.012511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/69ef2e0efb2143bab3ea9f3e12177331/roles/a98ede77714744cf8742d28cbcd62a5a` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.016746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=69ef2e0efb2143bab3ea9f3e12177331, role_id=a98ede77714744cf8742d28cbcd62a5a)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:39.018576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a0c78f-95d1-484d-8eac-80e8d10ba207 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:39.118725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 486/968] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:38 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/69ef2e0efb2143bab3ea9f3e12177331/roles/a98ede77714744cf8742d28cbcd62a5a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:39:39.602013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2954-7052-4411-875e-180883eae306 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.602529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2954-7052-4411-875e-180883eae306 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.603007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a2954-7052-4411-875e-180883eae306 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.605717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 483/969] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:39.614681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:39.615466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:39.615722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:39.646824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:39.648977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:39.733510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:39.744200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 487/970] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:39 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:40.086704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a2954-7052-4411-875e-180883eae306 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.159735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.215702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a2954-7052-4411-875e-180883eae306 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.255475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.256107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.256548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.256705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.257256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.259153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:40.268867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:40.270004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:40.270897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:40.279076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 484/971] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:40.279305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.398853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943c5c62-2b50-4c9d-9e6e-bd61aa6b8faf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.463419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.464474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.464975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.465469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.466947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.467544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.470285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.481297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 488/972] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.487399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.663921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499b988-4733-4bee-9660-9c5aad4607ac None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.701759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.729843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.730915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.731406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.731890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.733316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.733874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.736641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.751847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 485/973] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:40.755600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:40.828816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:40.881943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8da0828-743f-4438-9ff6-5b40d500f8eb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:40.921095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:40.921676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:40.921961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:40.922235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:40.923009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] RBAC: Adding query filter params (interface=admin, service_id=d17db9bf8871471982d5d7b5a6eccfe7, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:40.923316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:40.924701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cff748-0a25-4d66-aefb-61a3b4314707 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:40.936194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 489/974] 10.222.0.97 () {56 vars in 1249 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/endpoints?service_id=d17db9bf8871471982d5d7b5a6eccfe7&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:40.939319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:41.144683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b88158-f2b6-46ee-a461-8a4330880eb8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:41.209427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:41.210151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.210430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.210816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.212059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:41.214376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd3d05-a78c-4d8c-975d-1832add5dd79 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:41.230071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 486/975] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:40 2022] GET /identity/v3/services => generated 1727 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:41.530749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc978-8ae3-4d32-9d66-04d822e00910 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.531224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc978-8ae3-4d32-9d66-04d822e00910 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.531667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bc978-8ae3-4d32-9d66-04d822e00910 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.534252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 490/976] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:41 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:41.541624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.542115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.542605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.574171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:41.576083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:41.655243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0605e9-542e-4fb6-ac95-5d356c076565 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.655519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0605e9-542e-4fb6-ac95-5d356c076565 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.655767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0605e9-542e-4fb6-ac95-5d356c076565 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.657359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 491/977] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:41.660096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:41.663303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:41.663585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:41.663863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:41.685377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:41.686634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:41.696625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 487/978] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:41.776197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 492/979] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:41 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:41.969741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.080361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.097055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf9092-443e-49d8-b332-48a006c3f2a4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.150620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.151259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.151518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.151800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.163914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:42.169206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.170561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:42.171747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 488/980] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:41 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:42.176392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.177447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.218934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.225607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8800e732-7a8b-418d-93ca-22a96c266e17 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.264572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.265220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.265521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.265782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] PATH_INFO: `/v3/services/compute` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.266370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.267960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.277811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.278847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:42.280289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 493/981] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:42.287902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.301876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccb55c5-6a1e-46d1-bcb3-701245f043a8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.348498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.349118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.349390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.349660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.350338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.350640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.352371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.362422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 489/982] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.368993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.431323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c333742-99b2-4bf2-8579-b57a0a8c55ea None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.462697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.488256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.488256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.488256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.488256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.488887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.488887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.488887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.496380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 494/983] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.498741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.505677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.511786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab517cd-478e-490c-a6d2-a5c2e6988a9e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.552819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.553451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.553763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.554016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.569328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.571070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:42.587208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:42.588533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:42.589808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:42.589808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:42.589808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:42.589808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 490/984] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:42.594787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.635985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fbe87f-58e1-4f94-9c88-c35ef74c4042 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.688034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.689005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.690084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.690084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.690902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] RBAC: Adding query filter params (type=compute) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.691422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.693995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.702644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 495/985] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.708529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.741544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aa3c7a-f1b0-4c2d-8ff4-201e913b3980 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.794489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.802461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.803405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.803885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.804353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.805993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.806432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.808766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.837306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 491/986] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.838958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.860525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:42.896238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0033ed7f-87e3-4032-9b5f-d26ffe796f52 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:42.962040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:42.963119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:42.963627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:42.964129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:42.965551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] RBAC: Adding query filter params (interface=public, service_id=31bb6e941e974bee83430e9949c2751f, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:42.966121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:42.969095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:42.987451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 496/987] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/endpoints?service_id=31bb6e941e974bee83430e9949c2751f&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:42.989148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:42.993333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93e572b-3cfd-47a1-b184-daf32f908aa5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.031728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.032404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.032692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.032965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.041825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.043324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:43.056060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:43.057125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:43.058177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 492/988] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:43.066275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.136347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fe51b1-4de2-4bcb-ae29-92267b6f300e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.153035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.184033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.185301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.186091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.187803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.188612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.189882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.192992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.205528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 497/989] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:42 2022] GET /identity/v3/services => generated 1727 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:39:43.225820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3302063-df35-406e-8c13-93f778025ca6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.276842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.277784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.278278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.278760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.279717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:43.280273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.282058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.305908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 493/990] 10.222.0.97 () {56 vars in 1149 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:43.312616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.457217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332bce96-facb-4ace-9d9d-a957dc46133a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.500645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.501277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.501570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.501839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.564352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.599670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.601237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2710c92-869b-424c-b0c9-06c2c6022558 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.728124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 498/991] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:39:43.734400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.814598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd27f-933e-4306-aa4a-3d1b75d719e5 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.815408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd27f-933e-4306-aa4a-3d1b75d719e5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.815408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76afd27f-933e-4306-aa4a-3d1b75d719e5 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.816624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 499/992] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:43.822570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.822909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.823165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.843625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:43.844920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:43.860869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdb3627-bbda-4a9a-82e7-812c0025542f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:43.902913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:43.909868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:43.910517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:43.910845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:43.911108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:43.934390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:43.936097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:43.940202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 500/993] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:43.959334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:43.960690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:43.967286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:43.967286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:43.967286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:43.967286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 494/994] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:43.968586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:43.983884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.144642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1bd02a-a7e5-4c2a-bee8-2d56f1d1035f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.198198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.198830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.199116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.199394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.200054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.200386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.201766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.222913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 501/995] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.224967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.259931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.342190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.382468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806de00-7190-46a8-81cc-36d678fb647d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.407935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e354cf5-0a86-4238-8bab-a4c8c4f6d822 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.451126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.453370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.453370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.453370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.465599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.468495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.473281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.474010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.474472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.474994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.476013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.478582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:44.490250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.492064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:44.494893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:44.495773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:44.497562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 08 13:39:44.499971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:44.503069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 495/996] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:44.503069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.504690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 502/997] 10.222.0.97 () {56 vars in 1126 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:44.507219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.574710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.656991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfcdbb-bc35-4eac-99f8-d1fa43167c0e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.701356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc955c4-fc8e-4800-854d-e3e617c09d7a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.703850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.704661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.705045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.705329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.706025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.706325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.707785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.728748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 496/998] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:44.732220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.756104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.763069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.763951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.764373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.764842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.766018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.766790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.769979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.785165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 503/999] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:44.786857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:44.863233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4120cff-997f-4d3c-98d3-fe0f341b8f49 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:44.865986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:44.915409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:44.916339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:44.916617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:44.916887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:44.917794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] RBAC: Adding query filter params (role.id=a98ede77714744cf8742d28cbcd62a5a, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=69ef2e0efb2143bab3ea9f3e12177331) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:44.918991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:44.920490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:44.936661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 497/1000] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/role_assignments?user.id=69ef2e0efb2143bab3ea9f3e12177331&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=a98ede77714744cf8742d28cbcd62a5a => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:44.983397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16c9d8-9e43-4dd9-99d1-4baa14a8dc10 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.052823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.053852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.054352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.054853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.055894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:45.056477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.059109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e148e16-3879-4101-a337-9be8501d4963 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.074791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 504/1001] 10.222.0.97 () {56 vars in 1122 bytes} [Thu Sep 8 13:39:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:45.079095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:45.225732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c28770-a512-4e4d-bb6b-d7bbc4dd1842 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:45.270316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:45.270833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.271107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.271377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.272065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:45.273505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cc7da0-344b-443a-a753-61029c529fc3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:45.353539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 498/1002] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:45 2022] POST /identity/v3/endpoints => generated 325 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:39:45.634474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec86c6-0afb-42b9-91a5-c7cd1c92d80d None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.634711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec86c6-0afb-42b9-91a5-c7cd1c92d80d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.635126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cec86c6-0afb-42b9-91a5-c7cd1c92d80d None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.636589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 505/1003] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:45.642401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:45.642790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:45.642946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:45.663076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:45.664420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:45.689925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:45.763605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 499/1004] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:45 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:46.045902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec86c6-0afb-42b9-91a5-c7cd1c92d80d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.122958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.174733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cec86c6-0afb-42b9-91a5-c7cd1c92d80d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.232334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.233444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.233931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.234421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] PATH_INFO: `/v3/services/compute` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.235478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.238087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:46.256665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:46.258870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 08 13:39:46.262042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:46.265112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 506/1005] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:46.269232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.460448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2db88-91de-42c1-8df7-e2f5601c7e6d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.507376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.508433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.508870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.509313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.510585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] RBAC: Adding query filter params (name=compute) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.511248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.514001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.529986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 500/1006] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:46.532611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.656514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343ead29-c3b2-4b8c-9e37-1bd1fa23eef0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.687958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.701066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] RBAC: Adding query filter params (type=compute) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.716975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e128f6b8-5eb9-400e-b80b-b1f4603012ce None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:46.717439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 507/1007] 10.222.0.97 () {56 vars in 1112 bytes} [Thu Sep 8 13:39:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:46.717729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:46.740590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:46.843861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f7f481-1295-4f59-bb7b-cd79edbfc9b8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:46.887235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:46.887985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:46.888286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:46.888444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:46.889110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:46.890515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c6ec10-48e2-4a8f-a07a-0e4fada5157a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:47.041982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 501/1008] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:39:46 2022] POST /identity/v3/endpoints => generated 334 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:47.375021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de7c6-2832-458d-a547-cb2bdefb2274 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.375816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de7c6-2832-458d-a547-cb2bdefb2274 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.375969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1de7c6-2832-458d-a547-cb2bdefb2274 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.378769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 508/1009] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:47.386305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9ef6-bced-419c-8cdb-d08c3a7a1f98 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.387117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9ef6-bced-419c-8cdb-d08c3a7a1f98 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.387266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccc9ef6-bced-419c-8cdb-d08c3a7a1f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.412089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-eccc9ef6-bced-419c-8cdb-d08c3a7a1f98 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:47.413622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-eccc9ef6-bced-419c-8cdb-d08c3a7a1f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:47.514586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:47.528580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 502/1010] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:47 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:47.874959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a342bb-9475-483b-bf1a-cf02ae185945 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.875573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a342bb-9475-483b-bf1a-cf02ae185945 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.876061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a342bb-9475-483b-bf1a-cf02ae185945 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.878682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 509/1011] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:47.888488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:47.888927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:47.889229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:47.916263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:47.917660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:47.955018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a342bb-9475-483b-bf1a-cf02ae185945 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.031703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 503/1012] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:47 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:48.123781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a342bb-9475-483b-bf1a-cf02ae185945 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.193548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.194480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.194928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.195393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] PATH_INFO: `/v3/services/image` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.196591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.199111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:48.217255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:48.218357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:48.219255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:48.221748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 510/1013] 10.222.0.97 () {56 vars in 1105 bytes} [Thu Sep 8 13:39:47 2022] GET /identity/v3/services/image => generated 86 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:48.225974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.319628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.390117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.407387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b6c158-ae96-4db2-829d-407169f9f4c4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.457818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.471656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.472371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.472661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.472911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.473690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] RBAC: Adding query filter params (name=image) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.473883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.475344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.479310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd527c4f-16fd-48f3-9d30-f2e4a8ed5c50 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.482417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 504/1014] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.491776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.537583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.538242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.538533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.538795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.539996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.541470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.543094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 511/1015] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:48.553275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:48.634755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cd6dd1-8f3d-4767-a81d-399da89d51ed None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.675997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.676747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.677100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.677380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.678030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] RBAC: Adding query filter params (type=image) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:48.678322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.679726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11d8add-f740-42de-afe0-9ad5cfd8350e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.687476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 505/1016] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:48.690058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29507a1f-45c6-4b6b-bd76-cdb568fbf39e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:48.718476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:48.735849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:48.736532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:48.736820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:48.737124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:48.737772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:48.739372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4d0eab-e354-492f-8f59-7d24a39a0ba8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:48.848491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 512/1017] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:48 2022] POST /identity/v3/projects => generated 311 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:49.753045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ed44f-f047-4210-9b63-d98e19f7d74c None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.753634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ed44f-f047-4210-9b63-d98e19f7d74c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.754036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99ed44f-f047-4210-9b63-d98e19f7d74c None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.756847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 506/1018] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:49.767178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:49.767766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:49.768314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:49.825296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:49.827462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:49.954994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 513/1019] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:49 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:50.265547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ed44f-f047-4210-9b63-d98e19f7d74c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.350315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.412777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99ed44f-f047-4210-9b63-d98e19f7d74c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.469665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.470691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.471186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.471668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.491173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:50.501625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:50.503558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:50.511826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 507/1020] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:50.513267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.695300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb1f6-ecbe-4175-9f14-45f4331f85b5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.698547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:50.767107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:50.768196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:50.768771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:50.769286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:50.770540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:50.771086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:50.773342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:50.789662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 514/1021] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:50.799682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:50.966978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21315b-dee2-4014-90ca-82797593c0f0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:50.991238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.013721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.014380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.014644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.014918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.028745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.030458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-17fd46c6-54a2-45f8-8189-d5cb8eba1888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:51.054100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:51.055249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:51.057821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:51.057821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:51.057821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:51.057821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 508/1022] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:51.066668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.230065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded1fd5-534d-44ca-8c80-703e63c0fd37 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.230463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded1fd5-534d-44ca-8c80-703e63c0fd37 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.230641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded1fd5-534d-44ca-8c80-703e63c0fd37 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.232090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 509/1023] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:51.237544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929c5d-a1d6-49ef-883f-817229ed6e5f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.237857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929c5d-a1d6-49ef-883f-817229ed6e5f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.238132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11929c5d-a1d6-49ef-883f-817229ed6e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.243880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20469768-4570-46f5-bb2c-407cc9d61e12 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.258477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-11929c5d-a1d6-49ef-883f-817229ed6e5f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:51.260260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-11929c5d-a1d6-49ef-883f-817229ed6e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:51.302458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.304016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.304653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.305156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.306814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.307759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.309952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.337881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 515/1024] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:51.340126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.361389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 510/1025] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:51 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:51.393628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.403767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad432-f48c-4d8c-925c-223d93422ede None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.404298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad432-f48c-4d8c-925c-223d93422ede None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.416610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082ad432-f48c-4d8c-925c-223d93422ede None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.416610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 511/1026] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:51.417523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.418026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.418672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.454824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:51.456940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:51.525167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6caf8ab-4330-4a1e-9035-ca4998a13935 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.569610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.570432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.570773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.571052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.580280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.581825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.594020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 512/1027] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:51 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:51.596273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:51.597266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:51.598320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 516/1028] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/users/nova => generated 82 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:51.607364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.645601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.663516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.782326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4743b5-7972-43c9-8207-6a215b657629 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.801738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:51.825908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99127456-c0d8-4247-8424-d3a6ef53be1a None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:51.833237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.834234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.834737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.835227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.837408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:51.838027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.841020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.862904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 513/1029] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:51.869421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.872412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:51.873391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:51.873844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:51.874282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:51.876281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:51.878778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:51.884230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 517/1030] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:51.888133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:51.986195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.004918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84e8ab5-01bc-4e4e-89b5-6d20ec4937f1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.017484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe97b563-11c6-4274-b19b-60b062855306 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.055044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.056608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.056954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.057234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.057439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.057846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.058120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.058392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.059049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.060729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.140036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.141591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.141782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 518/1031] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:39:51 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:52.142505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.156433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.255948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.276589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 514/1032] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:51 2022] GET /identity/v3/auth/tokens => generated 1873 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:39:52.283480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.287564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0129720-1b04-4db2-8638-0343046ee1c3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.364608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.365405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.365838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.366299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.367661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.370864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.437578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b55db-8838-4143-bf4b-4e2f24802c29 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.489793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.490473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.490760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.491033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.494350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 519/1033] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:39:52 2022] POST /identity/v3/services => generated 242 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:39:52.505138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.506658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:52.524767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:52.525862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:52.526890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:52.526890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:52.526890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:52.526890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 515/1034] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:52.536009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.678937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.750268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f7d9a6-1e90-48b8-8def-cfe01f547f62 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:52.797904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:52.798879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:52.799313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:52.799754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:52.800598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:52.801069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:52.802610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:52.821957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 520/1035] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:52.823760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:52.884161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:52.983697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e65fc1-03fc-4707-9e1e-4234d8f230c4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.037194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.037833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.038111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.038386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.046633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.048205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:53.061653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:53.063517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:53.069822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 516/1036] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:53.072874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.245164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866cce81-83da-43ca-9df3-ec2dfd4a749e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.289972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.303327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.303849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.304115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.304431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.305101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.305403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.306870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd395297-2ab7-4bd6-9c10-54a5c48b519a None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.328777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 521/1037] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:53.333285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:53.539442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:53.547576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb8687-2051-410f-a830-cbe38b4b9324 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:53.609357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:53.609982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:53.610257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:53.610534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:53.611283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] RBAC: Adding query filter params (role.id=a98ede77714744cf8742d28cbcd62a5a, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:53.611594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:53.613237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc139a4-ff84-49c4-8e3f-9cb462e92cf1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:53.620777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 517/1038] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:53 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=a98ede77714744cf8742d28cbcd62a5a => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:54.562161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b2eb0-3a30-4b7e-a59a-cbad4d9bf55c None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.562919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b2eb0-3a30-4b7e-a59a-cbad4d9bf55c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.562919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b2eb0-3a30-4b7e-a59a-cbad4d9bf55c None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.564405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 522/1039] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:54.570190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2153d-2d69-4555-80f6-15caf598fb23 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:54.570951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2153d-2d69-4555-80f6-15caf598fb23 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:54.570951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c2153d-2d69-4555-80f6-15caf598fb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:54.591432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d2c2153d-2d69-4555-80f6-15caf598fb23 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:54.592938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d2c2153d-2d69-4555-80f6-15caf598fb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:54.710148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 518/1040] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:54 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:54.990204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b2eb0-3a30-4b7e-a59a-cbad4d9bf55c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.078175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f06d2-7c6d-4cb0-8228-49b4e87aa4bc None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.078481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f06d2-7c6d-4cb0-8228-49b4e87aa4bc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.078793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f06d2-7c6d-4cb0-8228-49b4e87aa4bc None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.080181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 519/1041] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:39:55.085820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.086103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.086401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.107649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:55.108970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:55.159483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b2eb0-3a30-4b7e-a59a-cbad4d9bf55c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.202164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.202832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.203130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.203387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.208390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 520/1042] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:39:55.216806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:55.221999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:55.223875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:39:55.224995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:55.228566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 523/1043] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:39:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:55.230522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.278294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.413069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2de5b-1f8d-4d13-94de-8c92a5c347cb None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.471604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.472248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.472529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.472798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.473562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:55.473861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.475320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.484981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 521/1044] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:55.485506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.491156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.553341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.620976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.633616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33922cd8-5981-42d8-a311-b5115a1de22f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.646440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fdc932-1e74-4e7d-8c66-4bcb9cc1b959 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.694268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.695033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.695322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.695594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] PATH_INFO: `/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.705383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.706034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.706336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.706580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] PATH_INFO: `/v3/services/image` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.707382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.708826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.715340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] RBAC: Authorizing `identity:get_user(user_id=cb9f2aa55a7b4d309f8478f04ab6f007)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.717155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.719562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 522/1045] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007 => generated 285 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:55.720452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:55.722309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:39:55.723962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 524/1046] 10.222.0.97 () {56 vars in 1105 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/services/image => generated 86 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:55.729793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.730253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.865513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998de20a-0366-415b-9652-e9c057f918ad None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.909804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74396d90-224c-429c-ac2b-c57b5e1399ac None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:55.918192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.919257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.919756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.920310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.920815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.921488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] RBAC: Adding query filter params (name=image) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:55.922576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:55.925284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a862b3a-2359-4f93-8795-9d3ee335f71d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:55.932882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 523/1047] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/services?name=image => generated 122 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a060-101d-46f9-b0ab-187fb1e4b69d None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a060-101d-46f9-b0ab-187fb1e4b69d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d973a060-101d-46f9-b0ab-187fb1e4b69d None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 524/1048] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:39:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a060-101d-46f9-b0ab-187fb1e4b69d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:55.953944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:55.954873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:55.954873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:55.954873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:55.954873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.049481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.050921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.061310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d973a060-101d-46f9-b0ab-187fb1e4b69d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.101000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.101806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.101977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.102128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.103332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] RBAC: Adding query filter params (type=image) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.104442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.105489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5519fc1e-08e2-499b-bcc0-e40664f1bee1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.111438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 525/1049] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:39:56.112849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.113134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.113407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.134263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:39:56.135427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:39:56.206777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 525/1050] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:39:55 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:39:56.207848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.220708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.228613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.235966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 526/1051] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:39:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:39:56.239760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.375290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4451fbb6-aec8-4dce-9fe2-a2e9003aa927 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.393153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73795b11-8c4a-4f78-816f-3bc5724930cc None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.439355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.440018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.440355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.440634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.441897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.443362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.448685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.449564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.450327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.450488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.450640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 527/1052] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.451372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.451535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] RBAC: Adding query filter params (interface=public, service_id=f398794f5917458fb20092b0336ef2ba, region_id=RegionOne) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.452062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.454793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.470187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 526/1053] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/endpoints?service_id=f398794f5917458fb20092b0336ef2ba&interface=public®ion_id=RegionOne => generated 194 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.471757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.549545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.585319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.597596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce72307-6a77-4725-b24a-190d86e2dc8d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.624114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a29f7fa-d16c-479c-8bf5-110534539242 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.639480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.640464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.640879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.641308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.642553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=a36b0720713d482799c540466ca07cd0, user.id=cb9f2aa55a7b4d309f8478f04ab6f007) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:56.643072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.645774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.663409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.664051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.664221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 528/1054] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/role_assignments?user.id=cb9f2aa55a7b4d309f8478f04ab6f007&scope.project.id=a36b0720713d482799c540466ca07cd0&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:56.664401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.664636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.665260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:56.665499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:56.666763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:56.674949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 527/1055] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/services => generated 1957 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:39:56.859871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb17ef89-af22-4889-8168-925cc198901a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:56.898058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:56.904352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:56.905002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:56.905277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:56.905544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:56.919969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:39:56.925052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:56.926915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:56.929393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 529/1056] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:56.934128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.078029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82467052-75fc-4a04-854d-c82cb03e4db8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.148777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.149739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.150437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.150916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.152199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.152726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.155207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.159818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.180187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 528/1057] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:39:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.184563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.391732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fbefee-f8f5-4901-ad4f-d721fe84efb3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.457989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.458907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.459351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.459743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.480722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.483571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:57.511851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:57.513245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:57.514668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:57.514668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:57.514668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:57.515632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 530/1058] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:57.522448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.650270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b058c7-9d8f-4bdf-8ef4-921e3b9cd8ae None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.692614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.693630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.693630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.693903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.694613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:57.694923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.696341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:57.710539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 529/1059] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:57.716564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:57.804985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:57.914755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb8b39f-bc1c-4d13-9b50-7b4f23c20eea None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:57.975130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:57.976220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:57.976691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:57.977163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] PATH_INFO: `/v3/users/nova` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:57.992881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:57.996183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:39:58.015503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.016807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:39:58.017776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:58.023574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 531/1060] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:39:57 2022] GET /identity/v3/users/nova => generated 82 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:39:58.025335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.097933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.148908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75e193-caa9-4502-9bf3-8854cb4a92f9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.185921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.186551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.186834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.187108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.187853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:58.188191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.189624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.210536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 530/1061] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.216703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.348618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59223880-bef9-4ca4-84e3-b945bb99cfba None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.351622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.399410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.400656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.401171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.401664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.425111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.426164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:39:58.455776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.456880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:39:58.457785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:39:58.457785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:39:58.457785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:58.463270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 532/1062] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:58.465172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.594311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dee7ed-9864-433e-a966-061fdf1398aa None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.615445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:58.637995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.638630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.638914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.639196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.639865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:58.640327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.641762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.656671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 531/1063] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:58.662528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:58.793616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf97e9f1-40f9-41d9-a277-449aaaedc9d7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:58.834986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:58.835631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:58.835908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:58.836204 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:58.844834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:58.846216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:39:58.858934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:39:58.859970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:39:58.860949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:39:58.864222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 533/1064] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:39:58.866433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.030410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a91f21-262a-49d7-9d9d-2c40fc0817f6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.092693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.093969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.094455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.094994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.096248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:39:59.096788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.099431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92201d86-8a98-4b5f-b2da-93f180884054 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.123742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 532/1065] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:39:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:39:59.133721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:39:59.168533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:39:59.320202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b94acd-4ad3-4069-83fb-dab8df1dd46a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:39:59.381536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:39:59.382214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:39:59.382578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:39:59.383015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/a98ede77714744cf8742d28cbcd62a5a` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:39:59.386481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=46beefbd50f34b43b82977125f31ffeb, role_id=a98ede77714744cf8742d28cbcd62a5a)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:39:59.389112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91de8235-a553-461b-9a0b-81d53794d711 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:39:59.497547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 534/1066] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:39:59 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/46beefbd50f34b43b82977125f31ffeb/roles/a98ede77714744cf8742d28cbcd62a5a => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:00.031193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba8c42-8294-46c2-b298-3ce643757539 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.031856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba8c42-8294-46c2-b298-3ce643757539 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.032185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba8c42-8294-46c2-b298-3ce643757539 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.033972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 533/1067] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:00.034743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47044-7b4a-413a-bb1e-00f499972b68 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.035540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47044-7b4a-413a-bb1e-00f499972b68 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.035841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c47044-7b4a-413a-bb1e-00f499972b68 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.037352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 535/1068] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:00.041819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a93d-6b32-4378-8239-095316208093 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.042315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a93d-6b32-4378-8239-095316208093 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.042794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c956a93d-6b32-4378-8239-095316208093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.046296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.046741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.047161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.070866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c956a93d-6b32-4378-8239-095316208093 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:00.073073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c956a93d-6b32-4378-8239-095316208093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:00.075603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:00.077679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:00.123647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.192344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 534/1069] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:00.199491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 536/1070] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:00 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:00.480010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956a93d-6b32-4378-8239-095316208093 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.497485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.512511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.659885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c956a93d-6b32-4378-8239-095316208093 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.671815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d44ad9c-ff65-4b99-8015-a9133972c3f3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.736486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.737575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.738052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.738566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] PATH_INFO: `/v3/services/image` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.739634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.742251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.743285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.743970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.744274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.744550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.757394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.760792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.762740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:00.764410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 535/1071] 10.222.0.97 () {56 vars in 1105 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/services/image => generated 86 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:00.772920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:00.773789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:00.774718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 537/1072] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:00.775996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.777259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.829203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.856194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:00.902812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f98ae52-ff99-478a-8f51-9d62c507b362 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.935703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f5ab68-1044-45f8-8288-9f694eb23e42 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:00.952087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.952845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.953130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.953411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.954093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.954399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.955839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.965969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 538/1073] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:00.972435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:00.981122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:00.981913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:00.982293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:00.982665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:00.983423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] RBAC: Adding query filter params (name=image) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:00.983829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:00.985367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:00.993219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 536/1074] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.003965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.147645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b342d44-0f24-481e-b7d3-d32450ba68d9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.150439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.158456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63002219-01ff-4c06-8425-76dbfc6420e0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.187695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.198738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.199547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.199933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.200257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] PATH_INFO: `/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.201482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] RBAC: Authorizing `identity:get_user(user_id=cb9f2aa55a7b4d309f8478f04ab6f007)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.203025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.204718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 539/1075] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:00 2022] GET /identity/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007 => generated 285 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:01.209262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.210311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.210698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.211125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.212339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] RBAC: Adding query filter params (type=image) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:01.213474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.214508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.216256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.232602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 537/1076] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/services?type=image => generated 350 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:01.235932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.383462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d379a-71ad-42e2-8ce8-34e9043891af None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.385921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b82a2b-d6aa-4476-9c1e-e20c8ad1618f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.435048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.435834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.436210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.436567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.437495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.439024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c438ffca-6a67-414b-adee-4b2dba11f7b5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.449835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.450801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.451285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.451762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.453772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.456475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.459452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 540/1077] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:01 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:01.469459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:01.542381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 538/1078] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:40:01 2022] POST /identity/v3/endpoints => generated 327 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:40:01.554323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:01.606791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca9a25a-96fa-4ac7-8a2e-c5e8a073ba52 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:01.662232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:01.662910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.663186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.663455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0/users/cb9f2aa55a7b4d309f8478f04ab6f007/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.665619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] RBAC: Authorizing `identity:create_grant(project_id=a36b0720713d482799c540466ca07cd0, user_id=cb9f2aa55a7b4d309f8478f04ab6f007, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:01.667127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8137776-540b-46bd-bb70-6e5606b0f13f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:01.777398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 541/1079] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:01 2022] PUT /identity/v3/projects/a36b0720713d482799c540466ca07cd0/users/cb9f2aa55a7b4d309f8478f04ab6f007/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:01.874693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ef1a3-bc50-4ad2-8e9f-bbda8f993952 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.874997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ef1a3-bc50-4ad2-8e9f-bbda8f993952 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.875249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711ef1a3-bc50-4ad2-8e9f-bbda8f993952 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.876799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 539/1080] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:01.883043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:01.883410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:01.883581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:01.904251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:01.905570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:02.020463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 542/1081] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:02.301522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ef1a3-bc50-4ad2-8e9f-bbda8f993952 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.325364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.438166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711ef1a3-bc50-4ad2-8e9f-bbda8f993952 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.477398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.478031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.478347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.478636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.489689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:02.495121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:02.497327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 08 13:40:02.499499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:02.501981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 540/1082] 10.222.0.97 () {56 vars in 1120 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:02.507549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.578391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.644134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12794b94-e9ba-47ff-8a4b-2535ff5eec47 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.687644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.688617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.689050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.689468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.690314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:02.690792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.692446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.704654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 543/1083] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:02.711066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:02.810611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:02.857645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0782b2e3-b977-44b5-a556-a6acfbac52ea None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:02.914056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:02.915147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:02.915642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:02.916752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:02.937094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:02.939878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:02.970025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:02.971082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:02.971978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:02.971978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:02.971978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:02.972698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 541/1084] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:02.982148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.121931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a14a9d-6542-4183-86e3-246028e7ad38 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.137784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.167131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.168402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.168909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.169329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.170156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.170663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.172749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.189302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 544/1085] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.195237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.327655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2484f74-2311-4756-8536-9bd5eb400ec0 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.376982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:03.390027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.392799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.392799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.392799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] PATH_INFO: `/v3/users/nova` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.408196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.411043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:40:03.437947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:03.439400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 08 13:40:03.441074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:03.444666 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 542/1086] 10.222.0.97 () {56 vars in 1093 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:03.448949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.619419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06fcbf9-9da1-4a68-964d-280555628bbc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.663810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.664361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.664645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.664920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:03.665684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:03.666012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:03.667488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e73079e-4a4c-4377-9284-c0ed7cb9ddc1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:03.696176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 545/1087] 10.222.0.97 () {56 vars in 1133 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:03.699432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:03.895282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0daef6-fa8a-41a2-aa84-3bcde325fb27 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:03.947875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:03.948485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:03.948767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:03.949048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.025515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.033988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1fab6-64ef-4454-a8e3-3900908919b8 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.034260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1fab6-64ef-4454-a8e3-3900908919b8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.034545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd1fab6-64ef-4454-a8e3-3900908919b8 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.036001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 546/1088] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:04.043612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10add8-7fb3-4eaa-9c63-39ec01a3d546 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.044133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10add8-7fb3-4eaa-9c63-39ec01a3d546 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.044578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10add8-7fb3-4eaa-9c63-39ec01a3d546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.049863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.051318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.077634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1b10add8-7fb3-4eaa-9c63-39ec01a3d546 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:04.079652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1b10add8-7fb3-4eaa-9c63-39ec01a3d546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:04.119843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.189474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 547/1089] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:04.190986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbdfd5-172c-4905-911b-886caf89fb74 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.191277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbdfd5-172c-4905-911b-886caf89fb74 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.196730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47dbdfd5-172c-4905-911b-886caf89fb74 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.196914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 543/1090] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:04.197001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 548/1091] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:04.198764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.201857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.202150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.202430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.225973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:04.227746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:04.307256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 549/1092] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:04.314734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.331983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c2fe2-5a0d-4093-ba50-c4fc46214861 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.372316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.373005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.373359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.373706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.386132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.387690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:04.415072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:04.417903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:04.419800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:04.419800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:04.419800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:04.420610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 544/1093] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:04.427600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.504890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.574922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.592399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99754b32-bf1f-4ae0-a679-a157c9324dbd None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.656300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bde883-1b10-47f5-a88b-2ef3dfa32d82 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.661193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.662258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.662758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.663214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.664393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:04.664913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.667641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.688666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 550/1094] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:04.689557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.698086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.698710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.698988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.699253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.709381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.779502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:04.781112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:04.845408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb07691-207b-49f0-8541-246cc8ec17a1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:04.886334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:04.887073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:04.887353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:04.887613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:04.900550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:04.905889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:04.906907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:04.907738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:04.908420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 551/1095] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:04.909314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.925405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:04.926004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 545/1096] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:04.926878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:04.988772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.037782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9460af-e20d-4b30-856a-016f25fb3944 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.072122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216116c7-4137-4ef4-8cab-729903d811be None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.085239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.085881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.086160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.086421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.095690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.097212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:40:05.110446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:05.112313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 552/1097] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:05.114976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.121415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.122042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.122314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.122587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.123284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.123583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.125337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.141378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 546/1098] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.141378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.223617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.268298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.270844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0e96dc-7b74-4ac5-9bc9-dc1521bd4812 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.271601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa7555-abdc-4523-96de-fe5416c466a3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.312939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.313709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.314159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.314557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.315511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.315946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.317298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.318081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.318373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.318805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.319274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.333405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.333755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 547/1099] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.334826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.336053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:05.358358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:05.360709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:40:05.362586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:05.362586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:40:05.362586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:05.362586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 553/1100] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:05.362586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.478532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b63d399-3b4e-49a9-ad84-3f2fd53c10af None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.503671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28eb450-d3d7-4634-a3dc-262063de652b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.526737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.527377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.527662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.527944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] PATH_INFO: `/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.529211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] RBAC: Authorizing `identity:get_user(user_id=cb9f2aa55a7b4d309f8478f04ab6f007)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.530670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.531565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.532559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 548/1101] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/users/cb9f2aa55a7b4d309f8478f04ab6f007 => generated 285 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.533213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.550157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.550775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.551051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.551337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.552462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] RBAC: Adding query filter params (role.id=a98ede77714744cf8742d28cbcd62a5a, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=46beefbd50f34b43b82977125f31ffeb) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.552952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.554902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.561922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 554/1102] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/role_assignments?user.id=46beefbd50f34b43b82977125f31ffeb&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=a98ede77714744cf8742d28cbcd62a5a => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.562810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:05.576572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.673314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b809ef90-7eb1-4aa3-91b4-eb58b7f82a95 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.710740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f437dd8-7bcb-40a3-9604-9005dbcd533e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:05.734268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.735268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.735727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.736169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.737445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:05.737904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.740567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:05.772174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 549/1103] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:05.781184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:05.781956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:05.782314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:05.782663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:05.855479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:05.879128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:05.879933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.045210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 555/1104] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:06.052403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.189408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.204036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdccfbd-27ef-4af7-a3d3-2bd491a7ab57 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.249254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.249902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.250189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.250463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.251765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.253269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57c0c42-5888-4ef9-b0fe-89d12c96085f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.255045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 550/1105] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:06.262752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:06.378435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:06.398287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f3c0c-bf65-4600-96c6-8cfab0f99e0c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:06.448273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:06.448861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:06.449138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:06.449404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:06.450141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=a36b0720713d482799c540466ca07cd0, user.id=cb9f2aa55a7b4d309f8478f04ab6f007) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:06.450882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:06.451886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e364bb4-47a7-408f-a469-23ccef4e8637 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:06.459128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 556/1106] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:06 2022] GET /identity/v3/role_assignments?user.id=cb9f2aa55a7b4d309f8478f04ab6f007&scope.project.id=a36b0720713d482799c540466ca07cd0&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:08.766845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc156-6e4a-46bc-8dcb-c43ddeb5685b None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.767452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc156-6e4a-46bc-8dcb-c43ddeb5685b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.767452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1bc156-6e4a-46bc-8dcb-c43ddeb5685b None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.769450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 551/1107] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:08.778608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:08.779154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:08.779154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:08.798449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:08.799655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:08.871613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 557/1108] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:09.149092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bc156-6e4a-46bc-8dcb-c43ddeb5685b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.273675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1bc156-6e4a-46bc-8dcb-c43ddeb5685b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.311877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.312536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.312819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.313090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.314362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.315817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9149b7-317d-4ca5-b4a7-5755bc123427 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.317796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 552/1109] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:09.323592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:09.412839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:09.435464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f77e9-cd6d-407f-baef-83c8c7f933d0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:09.471827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51a2acf-8409-4118-8572-558206679311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:09.472451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a2acf-8409-4118-8572-558206679311 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:09.472733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a2acf-8409-4118-8572-558206679311 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:09.473000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51a2acf-8409-4118-8572-558206679311 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:09.473680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a2acf-8409-4118-8572-558206679311 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:09.475099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51a2acf-8409-4118-8572-558206679311 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:09.651136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 558/1110] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:09 2022] POST /identity/v3/users => generated 286 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:11.947603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4a6a-472c-441b-89e1-1899da7597bf None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.948988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4a6a-472c-441b-89e1-1899da7597bf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.948988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3e4a6a-472c-441b-89e1-1899da7597bf None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.950286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 553/1111] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:11.956010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:11.956498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:11.956690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:11.982634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:11.984741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:12.067082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 559/1112] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:12.339638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e4a6a-472c-441b-89e1-1899da7597bf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.366982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.452893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3e4a6a-472c-441b-89e1-1899da7597bf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.490301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.490948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.491225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.491505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] PATH_INFO: `/v3/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.492988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] RBAC: Authorizing `identity:get_role(role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.494350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.496364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 554/1113] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/roles/ca6f073019a043d6826e5e9ef797a451 => generated 223 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:12.503199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.580855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.617583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2b1fa-be52-40c8-bfba-a9d58c9a1827 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.654484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.655120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.655397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.655668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] PATH_INFO: `/v3/users/5f7fef37ca9f4539bf0ead3327407d49` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:12.671306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] RBAC: Authorizing `identity:get_user(user_id=5f7fef37ca9f4539bf0ead3327407d49)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:12.673355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:12.675775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 560/1114] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/users/5f7fef37ca9f4539bf0ead3327407d49 => generated 286 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:12.684350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:12.791099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:12.801084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6a4e34-cfc2-4cc4-9abb-b97019422b50 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:12.856458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:12.856981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:12.857259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:12.857633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.002581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.004170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.098126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.114664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 555/1115] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:13.114853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.219111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9918964-742d-4dc8-bdc7-609d7968f03e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.245781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:13.257397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.258000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.258279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.258552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.260033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.262818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5622a70-691e-4e1e-8b8f-8459a76d6a6f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.264994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 561/1116] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:13.274616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:13.405034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba00448-07a7-4347-adba-81f81dd031b3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:13.464780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:13.465530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:13.465963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:13.466407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:13.467493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=a36b0720713d482799c540466ca07cd0, user.id=5f7fef37ca9f4539bf0ead3327407d49) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:13.467831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:13.470352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26718583-5e9a-423e-b5be-b809e5d5bbb1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:13.479948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 556/1117] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:13 2022] GET /identity/v3/role_assignments?user.id=5f7fef37ca9f4539bf0ead3327407d49&scope.project.id=a36b0720713d482799c540466ca07cd0&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:15.683621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285bd58-f782-4918-ac72-e068d6bd150b None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.684184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285bd58-f782-4918-ac72-e068d6bd150b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.684184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285bd58-f782-4918-ac72-e068d6bd150b None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.685521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 562/1118] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:15.691301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:15.691778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:15.691867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:15.715913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:15.717146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:15.770492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:15.789137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 557/1119] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:16.075072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285bd58-f782-4918-ac72-e068d6bd150b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.185742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285bd58-f782-4918-ac72-e068d6bd150b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.223364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.223988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.224296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.224564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] PATH_INFO: `/v3/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.225972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] RBAC: Authorizing `identity:get_role(role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.227339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.228710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 563/1120] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:16 2022] GET /identity/v3/roles/ca6f073019a043d6826e5e9ef797a451 => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:16.237319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.344528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049d81cd-7d54-405b-9de5-0f97dc094ab2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.380865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.381500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.381783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.382060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] PATH_INFO: `/v3/users/5f7fef37ca9f4539bf0ead3327407d49` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.383204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] RBAC: Authorizing `identity:get_user(user_id=5f7fef37ca9f4539bf0ead3327407d49)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.384674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.386287 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 558/1121] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:16 2022] GET /identity/v3/users/5f7fef37ca9f4539bf0ead3327407d49 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:16.394961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.461237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:16.509282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1916ef-c574-40ae-bf68-617712a8e1b0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.548352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.548989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.549270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.549544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.550693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.552209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18470e1-033c-4745-bb6e-78ab4c4ac19c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.553904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 564/1122] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:16 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:16.560806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:16.623257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:16.672492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bdbf3-914f-408a-8a03-30f20214de2b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:16.711567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:16.712657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:16.712657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:16.712969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0/users/5f7fef37ca9f4539bf0ead3327407d49/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:16.715457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a36b0720713d482799c540466ca07cd0, user_id=5f7fef37ca9f4539bf0ead3327407d49, role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:16.716945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5583c7b-8819-4356-a3d7-2435ffaec4b3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:16.794383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 559/1123] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:16 2022] PUT /identity/v3/projects/a36b0720713d482799c540466ca07cd0/users/5f7fef37ca9f4539bf0ead3327407d49/roles/ca6f073019a043d6826e5e9ef797a451 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:40:19.010541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299ba-ae2e-4c72-b6c7-06d3de088fb9 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.011685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299ba-ae2e-4c72-b6c7-06d3de088fb9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.011685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32299ba-ae2e-4c72-b6c7-06d3de088fb9 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.013006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 565/1124] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:19.019017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.019572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.019572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.039667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:19.040934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:19.115173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 560/1125] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:19.392423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32299ba-ae2e-4c72-b6c7-06d3de088fb9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.523950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32299ba-ae2e-4c72-b6c7-06d3de088fb9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.561377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.569591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.570171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.570447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.570776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] PATH_INFO: `/v3/roles/ca6f073019a043d6826e5e9ef797a451` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.572248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] RBAC: Authorizing `identity:get_role(role_id=ca6f073019a043d6826e5e9ef797a451)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.578241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.578241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 566/1126] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/roles/ca6f073019a043d6826e5e9ef797a451 => generated 223 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:19.581101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.631479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.689938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3275d6a8-e286-4217-9a4e-7bda2f43a437 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.732322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.732962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.733244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.733517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] PATH_INFO: `/v3/users/5f7fef37ca9f4539bf0ead3327407d49` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.734774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] RBAC: Authorizing `identity:get_user(user_id=5f7fef37ca9f4539bf0ead3327407d49)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.736224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:19.737872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 561/1127] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/users/5f7fef37ca9f4539bf0ead3327407d49 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:19.748760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:19.854185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818c36d0-8666-41d8-9349-2f73ded032f5 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:19.891387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:19.891994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:19.892302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:19.892575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:19.955920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:19.963156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:19.965358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.068239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 567/1128] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:40:20.075064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.195077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce10c641-caa0-4e7e-b06d-b8cb6efed63c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.209990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.231912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.232707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.232981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.233250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] PATH_INFO: `/v3/projects/a36b0720713d482799c540466ca07cd0` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.234373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] RBAC: Authorizing `identity:get_project(project_id=a36b0720713d482799c540466ca07cd0)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.235787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7bb8ad-d384-4a07-90e5-9064f01040d9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.237463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 562/1129] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/projects/a36b0720713d482799c540466ca07cd0 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:20.243889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:20.346878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:20.353666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458e4304-4827-46c0-8502-4c13b1ab2110 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:20.411905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:20.412615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:20.412941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:20.413255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:20.414036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] RBAC: Adding query filter params (role.id=ca6f073019a043d6826e5e9ef797a451, scope.project.id=a36b0720713d482799c540466ca07cd0, user.id=5f7fef37ca9f4539bf0ead3327407d49) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:20.414386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:20.415917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ba7a2f-919f-477d-a235-5d014b22b969 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:20.424740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 568/1130] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:20 2022] GET /identity/v3/role_assignments?user.id=5f7fef37ca9f4539bf0ead3327407d49&scope.project.id=a36b0720713d482799c540466ca07cd0&role.id=ca6f073019a043d6826e5e9ef797a451 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:22.671159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ce7d-32f2-4eea-8664-ff942957404b None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.671694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ce7d-32f2-4eea-8664-ff942957404b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.671694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e437ce7d-32f2-4eea-8664-ff942957404b None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.673246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 563/1131] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:22.679366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:22.679905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:22.679905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:22.700592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:22.701815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:22.775259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 569/1132] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:23.048767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437ce7d-32f2-4eea-8664-ff942957404b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.157729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e437ce7d-32f2-4eea-8664-ff942957404b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.193168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.193796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.194070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.194341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.195506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.196969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d839b66-cd2d-4322-9a3d-26b795d7d771 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.198571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 564/1133] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:23.209048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:23.307878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:23.319192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a39bdfd-6b15-41fa-9022-7b4bccbe99a5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:23.358467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:23.359091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:23.359372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:23.359642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:23.360335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:23.361784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9921ad4f-174b-442b-aa90-ccfd550b8e24 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:23.470641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 570/1134] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:23 2022] POST /identity/v3/projects => generated 311 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:25.797875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d543a7-ba56-4445-818e-54f63e132eb1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.798484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d543a7-ba56-4445-818e-54f63e132eb1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.798598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d543a7-ba56-4445-818e-54f63e132eb1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.800173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 565/1135] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:25.805987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:25.806529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:25.806529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:25.829856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:25.831091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:25.929700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 571/1136] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:26.204913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d543a7-ba56-4445-818e-54f63e132eb1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:26.264531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:26.327797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d543a7-ba56-4445-818e-54f63e132eb1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:26.365620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:26.366257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:26.366535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:26.366812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:26.368003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:26.369510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaaffa4-04d0-4b96-880c-4ce13dec2eb6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:26.371792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 566/1137] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:40:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:26.377984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:26.483506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:26.517596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f591aa-1d09-44a3-8b3d-2ad4d5d73e73 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:26.557481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:26.558164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:26.559119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:26.559580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:26.560362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:26.561962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05240-ac52-4844-ac4a-6fc770d2b464 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:26.656219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 572/1138] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:26 2022] POST /identity/v3/users => generated 286 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:28.958242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090fa1-0b00-41f1-965e-c3c33682401b None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.958842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090fa1-0b00-41f1-965e-c3c33682401b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.958842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d090fa1-0b00-41f1-965e-c3c33682401b None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.960199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 567/1139] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:28.966398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:28.966924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:28.966924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:28.988115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:28.989363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:29.066367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 573/1140] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:29.336451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090fa1-0b00-41f1-965e-c3c33682401b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:29.444574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d090fa1-0b00-41f1-965e-c3c33682401b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:29.462886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:29.482421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:29.483063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:29.483345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:29.483619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:29.495313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:29.500951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:29.502272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:29.503579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 568/1141] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:29.508616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:29.570805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:29.614636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd5c70-2cc2-4953-99e3-f90398980dd4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:29.650429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:29.651029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:29.651311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:29.651586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:29.652280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:29.652585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:29.653948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:29.662325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 574/1142] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:29.668885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:29.776122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d0190-560b-4d1f-bd2a-d893a27c7402 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:29.811850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:29.812505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:29.812796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:29.813074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] PATH_INFO: `/v3/users/19af664d42754666abf5791c3ae73ebc` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:29.827653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] RBAC: Authorizing `identity:get_user(user_id=19af664d42754666abf5791c3ae73ebc)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:29.829047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:29.830732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 569/1143] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:29 2022] GET /identity/v3/users/19af664d42754666abf5791c3ae73ebc => generated 286 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:29.836933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:29.953734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7356f2a7-709a-4f06-9457-27cbd0bf61da None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:29.974578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:29.989458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:29.990110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:29.990388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:29.990664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.059302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:30.060798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:30.163156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 575/1144] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:30.169189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:30.207131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:30.275894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d987b6c-fd60-46b0-abfa-6761e9223d38 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:30.313267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:30.313898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:30.314175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:30.314450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] PATH_INFO: `/v3/projects/4bd5227e2b2d459593a86ae18d64ba37` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.315644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] RBAC: Authorizing `identity:get_project(project_id=4bd5227e2b2d459593a86ae18d64ba37)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:30.317122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908edf0-2101-4418-9d6a-4c094a641e73 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:30.318765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 570/1145] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:30 2022] GET /identity/v3/projects/4bd5227e2b2d459593a86ae18d64ba37 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:30.325517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:30.349737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:30.438974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629011a7-d19b-48d6-bf7f-6ce1f421dcec None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:30.477532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:30.478133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:30.478410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:30.478679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:30.479473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=4bd5227e2b2d459593a86ae18d64ba37, user.id=19af664d42754666abf5791c3ae73ebc) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:30.479775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:30.481764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bae9246-96dc-4326-9268-7449c341daf6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:30.487971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 576/1146] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:30 2022] GET /identity/v3/role_assignments?user.id=19af664d42754666abf5791c3ae73ebc&scope.project.id=4bd5227e2b2d459593a86ae18d64ba37&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:32.771016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b6f8-9c05-4421-a516-b1d6a971a8f3 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.771016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b6f8-9c05-4421-a516-b1d6a971a8f3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.771016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b7b6f8-9c05-4421-a516-b1d6a971a8f3 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.771016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 571/1147] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:32.771810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:32.771810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:32.771810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:32.771810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:32.771810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:32.837472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 577/1148] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:33.116876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7b6f8-9c05-4421-a516-b1d6a971a8f3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.230408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b7b6f8-9c05-4421-a516-b1d6a971a8f3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.232082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.265644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.266268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.266545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.266816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.277485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:33.282823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:33.284050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:33.285096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 572/1149] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:33.290068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.305049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.392766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fa3f7d-c332-4ec4-9f48-08487a28a809 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.429068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.429675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.429949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.430215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.430870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:33.431169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.432567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.442172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 578/1150] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.448127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.560443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355a989-7ac2-4556-bacd-06456500c264 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.599044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.599682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.599963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.600265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] PATH_INFO: `/v3/users/19af664d42754666abf5791c3ae73ebc` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.601413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] RBAC: Authorizing `identity:get_user(user_id=19af664d42754666abf5791c3ae73ebc)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.602839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.604459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 573/1151] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/users/19af664d42754666abf5791c3ae73ebc => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.611138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.697483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.721476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879d5ef9-5ef0-4c45-85dc-7ba001ec6b00 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.755963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.756639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.756919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.757198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] PATH_INFO: `/v3/projects/4bd5227e2b2d459593a86ae18d64ba37` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.758317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] RBAC: Authorizing `identity:get_project(project_id=4bd5227e2b2d459593a86ae18d64ba37)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.759759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ba98d-8bd2-4e62-9c25-6d928b729c99 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:33.761444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 579/1152] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:33 2022] GET /identity/v3/projects/4bd5227e2b2d459593a86ae18d64ba37 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:33.767623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:33.801685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:33.878521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3c99ed-2cbe-4d4d-a7ec-b19de2996cbc None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:33.920604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:33.921263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:33.921546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:33.921818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] PATH_INFO: `/v3/projects/4bd5227e2b2d459593a86ae18d64ba37/users/19af664d42754666abf5791c3ae73ebc/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:33.923769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] RBAC: Authorizing `identity:create_grant(project_id=4bd5227e2b2d459593a86ae18d64ba37, user_id=19af664d42754666abf5791c3ae73ebc, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:33.925251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb06ef12-e7a5-4753-9b33-385873539348 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:34.027495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 574/1153] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:33 2022] PUT /identity/v3/projects/4bd5227e2b2d459593a86ae18d64ba37/users/19af664d42754666abf5791c3ae73ebc/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:36.261355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999330a3-b199-49b9-a5c3-fa81eb7f4aa4 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.262002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999330a3-b199-49b9-a5c3-fa81eb7f4aa4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.262002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999330a3-b199-49b9-a5c3-fa81eb7f4aa4 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.264119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 580/1154] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:36.270747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.271356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.271356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.291483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:36.292746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:36.366407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 575/1155] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:40:36.640472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999330a3-b199-49b9-a5c3-fa81eb7f4aa4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.754217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999330a3-b199-49b9-a5c3-fa81eb7f4aa4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.790162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.790792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.791070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.791349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.803295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:36.808429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:36.810918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:36.812539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 581/1156] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:40:36.816416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.839383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:36.930587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35385b1f-69fb-47c6-a841-9e9e79d95456 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:36.973052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:36.973671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:36.973952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:36.974292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:36.975147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:36.975451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:36.976978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:36.987416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 576/1157] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:36.993821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:36.998485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.109809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34d5ab6-7c21-4f70-a12d-fbfb3f187ceb None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.148034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.148692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.148971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.149241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] PATH_INFO: `/v3/users/19af664d42754666abf5791c3ae73ebc` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.150366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] RBAC: Authorizing `identity:get_user(user_id=19af664d42754666abf5791c3ae73ebc)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.151790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.154100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 582/1158] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:36 2022] GET /identity/v3/users/19af664d42754666abf5791c3ae73ebc => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:37.161360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.251926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.272196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e80d8-c26a-4189-a39a-798c76cac8f3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.316002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.316673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.316951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.317223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.398258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.399714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.498609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.520648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 577/1159] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:37.527141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.607809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:37.663122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044413-886e-4b7b-9e78-762bb90577e2 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.723637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.724760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.725273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.725769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] PATH_INFO: `/v3/projects/4bd5227e2b2d459593a86ae18d64ba37` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.727814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] RBAC: Authorizing `identity:get_project(project_id=4bd5227e2b2d459593a86ae18d64ba37)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.730572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c43013-1c54-48bb-883d-e0caa24949a0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.734308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 583/1160] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/projects/4bd5227e2b2d459593a86ae18d64ba37 => generated 311 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:37.741211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:37.862201 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99952ccb-6810-4735-ae07-e26a3a287365 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:37.901071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:37.901679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:37.901957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:37.902228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:37.903687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=4bd5227e2b2d459593a86ae18d64ba37, user.id=19af664d42754666abf5791c3ae73ebc) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:37.904109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:37.905589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9bb0a1-f462-4754-9f27-6fcc7f08c694 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:37.912475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 578/1161] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:37 2022] GET /identity/v3/role_assignments?user.id=19af664d42754666abf5791c3ae73ebc&scope.project.id=4bd5227e2b2d459593a86ae18d64ba37&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:40.154486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c14f-2e6f-4470-8da4-e6537fab1d39 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.155030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c14f-2e6f-4470-8da4-e6537fab1d39 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.155030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8366c14f-2e6f-4470-8da4-e6537fab1d39 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.156377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 584/1162] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:40.162722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.163222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.163532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.186136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:40.188257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:40.267656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.308589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 579/1163] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:40.579577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366c14f-2e6f-4470-8da4-e6537fab1d39 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:40.684514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8366c14f-2e6f-4470-8da4-e6537fab1d39 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:40.720807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:40.721405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:40.721686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:40.721963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:40.733391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:40.789718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:40.791189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:40.896650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 585/1164] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:40.902610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.003744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cc057-22c6-47be-b2c0-ab36d20babdf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.038047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.038691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.038968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.039243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.047544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.049015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-05b31a86-9c0e-4c72-857e-0e2ccca0a13d None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:41.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:41.062724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:40:41.064398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 580/1165] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:40:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:41.068665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:41.114875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:41.176035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2427aca2-f09d-4b59-ac58-2293e1e605c3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:41.212287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:41.213256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:41.213537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:41.213810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:41.214461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:41.214764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:41.216714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9b90a0-2be0-4e2f-b8de-2e9eef767866 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:41.224810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 586/1166] 10.222.0.97 () {56 vars in 1115 bytes} [Thu Sep 8 13:40:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:43.412492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd2ae-bd6a-49da-afae-d3a916b07df1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.413138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd2ae-bd6a-49da-afae-d3a916b07df1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.413138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efdd2ae-bd6a-49da-afae-d3a916b07df1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.414432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 581/1167] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:43.421161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d94-5e49-4986-917f-5368dfde4029 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.421603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d94-5e49-4986-917f-5368dfde4029 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.421708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e2d94-5e49-4986-917f-5368dfde4029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.440680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-065e2d94-5e49-4986-917f-5368dfde4029 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:43.441888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-065e2d94-5e49-4986-917f-5368dfde4029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:43.533994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 587/1168] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:43 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:43.802386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd2ae-bd6a-49da-afae-d3a916b07df1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:43.841759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:43.927342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efdd2ae-bd6a-49da-afae-d3a916b07df1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:43.968707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:43.969380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:43.969661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:43.969940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:43.970659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:43.972111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326b0219-7a96-40ed-8515-fb1ebbe6336d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:44.043105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 582/1169] 10.222.0.97 () {60 vars in 1138 bytes} [Thu Sep 8 13:40:43 2022] POST /identity/v3/domains => generated 267 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:46.354200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0fd-d2f8-43a0-8db3-011dd04145f5 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:46.354827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0fd-d2f8-43a0-8db3-011dd04145f5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:46.354827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f7d0fd-d2f8-43a0-8db3-011dd04145f5 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:46.356123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 588/1170] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:46.362675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:46.363215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:46.363215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:46.382593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:46.383813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:46.460048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 583/1171] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:46.735039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d0fd-d2f8-43a0-8db3-011dd04145f5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:46.796753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:46.891743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f7d0fd-d2f8-43a0-8db3-011dd04145f5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:46.928849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:46.929486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:46.929760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:46.930035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] PATH_INFO: `/v3/domains/096cc5ea716e474d87e30616be00a45e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:46.941763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=096cc5ea716e474d87e30616be00a45e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:46.943213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b067a83c-618e-4ac0-a067-12d43ef27ad6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:46.944914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 589/1172] 10.222.0.97 () {56 vars in 1183 bytes} [Thu Sep 8 13:40:46 2022] GET /identity/v3/domains/096cc5ea716e474d87e30616be00a45e => generated 267 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:46.954005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:46.991288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:47.065281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ec17a-5bf2-4e41-8335-148312ea352b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:47.102699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:47.103312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:47.103590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:47.103862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:47.104555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:47.105963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2645d727-16ca-4ade-85a6-a6a843f5de15 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:47.284895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 584/1173] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:40:46 2022] POST /identity/v3/projects => generated 361 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:49.558358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823cc1-c81b-42fe-a81d-98faeba3cb04 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:49.558966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823cc1-c81b-42fe-a81d-98faeba3cb04 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:49.558966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42823cc1-c81b-42fe-a81d-98faeba3cb04 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:49.561184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 590/1174] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:49.569341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:49.569909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:49.569909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:49.590288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:49.591545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:49.664940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 585/1175] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:49.935040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42823cc1-c81b-42fe-a81d-98faeba3cb04 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:50.037425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:50.045593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42823cc1-c81b-42fe-a81d-98faeba3cb04 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:50.083086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:50.083717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:50.083988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:50.084287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] PATH_INFO: `/v3/domains/096cc5ea716e474d87e30616be00a45e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:50.085397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=096cc5ea716e474d87e30616be00a45e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:50.086816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f825d130-e0c7-49a4-949c-0aeb22f685cc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:50.088408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 591/1176] 10.222.0.97 () {56 vars in 1183 bytes} [Thu Sep 8 13:40:49 2022] GET /identity/v3/domains/096cc5ea716e474d87e30616be00a45e => generated 267 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:50.096756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:50.107976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:50.204299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590b2542-f2e5-4a8c-9af8-0462388ccd26 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:50.242468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:50.243087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:50.243372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:50.243649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:50.244341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:50.245764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33ca389-05ca-4ff5-bdee-671c65e51862 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:50.374176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 586/1177] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:40:50 2022] POST /identity/v3/users => generated 311 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:40:52.660359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb7fb-8884-4ed2-9800-2f2cc12a5702 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:52.660896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb7fb-8884-4ed2-9800-2f2cc12a5702 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:52.660896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb7fb-8884-4ed2-9800-2f2cc12a5702 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:52.662150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 592/1178] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:52.667798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:52.668341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:52.668444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:52.687208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:52.688473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:52.751400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:52.759618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 587/1179] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:53.042306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bb7fb-8884-4ed2-9800-2f2cc12a5702 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:53.147314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bb7fb-8884-4ed2-9800-2f2cc12a5702 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:53.181437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:53.182065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.182343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.182623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.193743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:53.198550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:53.200363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:53.201465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 593/1180] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:53.206841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:53.343002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f4bb1-02c4-4468-9167-f103ebbfeeb4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:53.379329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:53.379942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.380253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.380534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.381209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:53.381516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:53.382880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:53.392686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 588/1181] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:53.398856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:53.427231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:53.509384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d72e61c-d49b-4163-80ae-712a56ba6cfb None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:53.544747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:53.545384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.545664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.545938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] PATH_INFO: `/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.560322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] RBAC: Authorizing `identity:get_user(user_id=d73fcd1a8ee2434386e14e6cfd3d5133)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:53.561782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:53.563513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 594/1182] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:53 2022] GET /identity/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133 => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:53.572621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:53.634927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:53.685447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914575d9-4842-4d4a-b8f8-2706e034a873 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:53.722682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:53.723301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:53.723584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:53.723857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:53.799164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:53.800654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:53.860656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:53.908388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 589/1183] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:40:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:40:53.914272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.002668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:54.026425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc48fea-8dad-4ac7-a653-a768dd784f1a None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.068085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.068742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.069018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.069290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] PATH_INFO: `/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.070409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] RBAC: Authorizing `identity:get_project(project_id=1590ad84f0374b5c8303af0c4d0bd8ad)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.071831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951f3c41-fafb-4f84-91dc-7dbba002e555 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.073500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 595/1184] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:53 2022] GET /identity/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:54.080642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:54.179714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877f4702-684a-4f3f-9e27-cab1fcf41e21 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:54.215949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:54.217656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:54.217656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:54.217656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:54.218153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=1590ad84f0374b5c8303af0c4d0bd8ad, user.id=d73fcd1a8ee2434386e14e6cfd3d5133) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:54.218457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:54.219907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb6632-766b-4a4a-8471-0f2407919587 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:54.226456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 590/1185] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:40:54 2022] GET /identity/v3/role_assignments?user.id=d73fcd1a8ee2434386e14e6cfd3d5133&scope.project.id=1590ad84f0374b5c8303af0c4d0bd8ad&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:56.447104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693440-e03f-4fa9-8c5f-c3315237285a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:56.448278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693440-e03f-4fa9-8c5f-c3315237285a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:56.448438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99693440-e03f-4fa9-8c5f-c3315237285a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:56.449713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 596/1186] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:40:56.455537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:56.456097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:56.456097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:56.475812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:56.477072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:40:56.492921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:56.553636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 591/1187] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:40:56.824124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99693440-e03f-4fa9-8c5f-c3315237285a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:56.929994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99693440-e03f-4fa9-8c5f-c3315237285a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:56.965725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:56.966347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:56.966625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:56.966896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:56.977871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:40:56.983323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:40:56.984690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:40:56.985546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 597/1188] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:40:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:40:56.991282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.094004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f800ef81-9a7c-4443-9769-4b22c9f7a704 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.130009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.130617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.130901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.131176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.131845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:40:57.132185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.133523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.142412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 592/1189] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:40:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:40:57.148700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.160693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.259192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a870a1-5bd6-49fd-b187-9f2f6e36059b None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.297163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.297794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.298073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.298345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] PATH_INFO: `/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.299503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=d73fcd1a8ee2434386e14e6cfd3d5133)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.300964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.302559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 598/1190] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133 => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:57.308597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.326409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.414342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caf732-09b8-45c0-a1e2-a59114f7c6c1 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.453050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.453688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.454011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.454284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] PATH_INFO: `/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.455429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] RBAC: Authorizing `identity:get_project(project_id=1590ad84f0374b5c8303af0c4d0bd8ad)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.456881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741bc52f-87cc-4bf9-93a1-66982f4ea6ef None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.458631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 593/1191] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:40:57 2022] GET /identity/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:40:57.465604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:40:57.547701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:40:57.572942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80db8c80-35e9-48ea-917f-7cb889222ca0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:40:57.614731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:40:57.615834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:57.616367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:57.616863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] PATH_INFO: `/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad/users/d73fcd1a8ee2434386e14e6cfd3d5133/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:57.620759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] RBAC: Authorizing `identity:create_grant(project_id=1590ad84f0374b5c8303af0c4d0bd8ad, user_id=d73fcd1a8ee2434386e14e6cfd3d5133, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:40:57.623471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b960cb-c6e1-4e14-a5d3-50a3f6eaab40 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:40:57.704162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 599/1192] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:40:57 2022] PUT /identity/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad/users/d73fcd1a8ee2434386e14e6cfd3d5133/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:40:59.969308 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e3fe0-ce4f-4a6e-9196-bc2403577d14 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:59.969926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e3fe0-ce4f-4a6e-9196-bc2403577d14 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:59.969926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68e3fe0-ce4f-4a6e-9196-bc2403577d14 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:59.971183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 594/1193] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:40:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:40:59.976838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:40:59.977369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:40:59.977369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:40:59.996393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:40:59.997610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:41:00.068577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 600/1194] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:40:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:41:00.345896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e3fe0-ce4f-4a6e-9196-bc2403577d14 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:00.434042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:00.454788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68e3fe0-ce4f-4a6e-9196-bc2403577d14 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:00.490982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:00.491623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.491939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.492736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.504102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:41:00.509068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:41:00.510289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:41:00.511237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 595/1195] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:41:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:41:00.516718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:00.588436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:00.630485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1086be6-6d56-491a-82e7-95ea274e3e9f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:00.683590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:00.684673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.685158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.685633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.686845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:00.687401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:00.690006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:00.707084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 601/1196] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:41:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:00.713583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:00.885837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7027faf-5728-447e-b841-fb9b1686f3eb None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:00.924577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:00.925272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:00.925548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:00.925822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] PATH_INFO: `/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:00.926998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] RBAC: Authorizing `identity:get_user(user_id=d73fcd1a8ee2434386e14e6cfd3d5133)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:00.928447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:00.930610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 596/1197] 10.222.0.97 () {56 vars in 1177 bytes} [Thu Sep 8 13:41:00 2022] GET /identity/v3/users/d73fcd1a8ee2434386e14e6cfd3d5133 => generated 311 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:41:00.936619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:01.055796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a96715-8b55-47f9-9390-858bf11f1cef None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:01.086434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:01.094914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:01.098684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:01.098684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:01.098684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:01.179820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:01.181310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:01.284047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 602/1198] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:41:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:41:01.290830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:01.302213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:01.425800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4019df0f-1826-4170-b3eb-e6cf867b4288 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:01.478706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:01.479339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:01.479617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:01.479898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] PATH_INFO: `/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:01.481063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] RBAC: Authorizing `identity:get_project(project_id=1590ad84f0374b5c8303af0c4d0bd8ad)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:01.482494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806b744-566a-48ad-9a1b-cb7d1fe3c8fb None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:01.484159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 597/1199] 10.222.0.97 () {56 vars in 1186 bytes} [Thu Sep 8 13:41:01 2022] GET /identity/v3/projects/1590ad84f0374b5c8303af0c4d0bd8ad => generated 361 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:01.490617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:41:01.518346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:01.590771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51f10-5a1e-4a86-ad23-0cc524604ea4 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:41:01.626004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:41:01.626600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:41:01.626878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:41:01.627152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:41:01.627864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=1590ad84f0374b5c8303af0c4d0bd8ad, user.id=d73fcd1a8ee2434386e14e6cfd3d5133) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:41:01.628198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:41:01.629699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a2156e-e55f-4092-b7c3-7441c49b81f2 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:41:01.635982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 603/1200] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:41:01 2022] GET /identity/v3/role_assignments?user.id=d73fcd1a8ee2434386e14e6cfd3d5133&scope.project.id=1590ad84f0374b5c8303af0c4d0bd8ad&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:41:11.473572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=62 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:41:11.633448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=54 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:24.566940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433579-4273-4b23-86e2-208f8b67ce82 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.567597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433579-4273-4b23-86e2-208f8b67ce82 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.567597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47433579-4273-4b23-86e2-208f8b67ce82 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.569249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 598/1201] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:24.575433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:24.576247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:24.576247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:24.598944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:24.600868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:24.683215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 604/1202] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:24.967846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433579-4273-4b23-86e2-208f8b67ce82 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:25.130600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47433579-4273-4b23-86e2-208f8b67ce82 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:25.170790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:25.171483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:25.171802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:25.172129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:25.184855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:25.186377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:25.204639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:25.207091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:25.209070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:25.209070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:25.209070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:25.209070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 599/1203] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:25.214361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:25.381805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4107ea-a413-49eb-95fa-6a79c1ac186d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:25.433996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:25.441458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:25.442153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:25.442490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:25.442799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:25.443501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:25.443838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:25.445354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6656b2-0fc9-41d8-a39d-34b5cc1c22f4 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:25.461828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 605/1204] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:25.469124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:25.533674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:25.590464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc717e-e859-4394-8abc-3a63812d3b1f None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:25.631500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:25.632185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:25.632469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:25.632748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:25.633465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:25.634896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db04a1bc-d8ed-49cb-8434-d6c4dd1b294e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:25.744381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 600/1205] 10.222.0.97 () {60 vars in 1132 bytes} [Thu Sep 8 13:42:25 2022] POST /identity/v3/users => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:42:28.162745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c08bd-eb1d-458c-b88d-2fe8645cfa7e None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:28.163625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c08bd-eb1d-458c-b88d-2fe8645cfa7e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:28.163625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53c08bd-eb1d-458c-b88d-2fe8645cfa7e None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:28.164693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 606/1206] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:28.171133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:28.171748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:28.171748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:28.193612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:28.194915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:28.295099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 601/1207] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:28.568670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c08bd-eb1d-458c-b88d-2fe8645cfa7e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:28.696792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53c08bd-eb1d-458c-b88d-2fe8645cfa7e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:28.760340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:28.761436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:28.761907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:28.762422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:28.784062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:28.794354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:28.795651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:28.796718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:28.797758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 607/1208] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:28.803755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:28.856622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:28.929244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d680c-d535-4c65-ba85-ad01effcb9ef None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:28.977663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:28.978672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:28.979277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:28.979660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:28.980553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:28.980854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:28.982247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:28.992110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 602/1209] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:29.001506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:29.097284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:29.186821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef686ccd-af4c-4be6-8cb6-12ec7c4ab882 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:29.229493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:29.230138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.230414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.230683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.243892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:29.245416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:29.262247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:29.263917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:29.264915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:29.264915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:29.264915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:29.264915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 608/1210] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:29.273714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:29.465739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:29.469125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df322b15-45c5-46fc-aac0-971c4401c35e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:29.509431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:29.510042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.510319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.510589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.511236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:29.511536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:29.512945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:29.526581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 603/1211] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:29.536356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:29.659038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7579f757-4278-4e7d-83a7-4d88f78caff8 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:29.661175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:29.702785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:29.704097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.704474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.704798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] PATH_INFO: `/v3/users/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.715538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:29.717126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:29.733699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:29.739352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:29.741305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 609/1212] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:29.743633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:29.876015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d133be21-61e7-4470-b008-0f036328a1a5 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:29.916298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:29.917030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:29.917424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:29.917792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:29.918627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:29.919020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:29.920611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:29.950015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 604/1213] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:29.956851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:30.119483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea076843-be2f-4c3b-9388-b2e2a5736adf None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:30.182414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:30.183669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:30.184127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:30.184637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:30.228837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:30.276975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:30.278605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:30.401447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 610/1214] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:30.408410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:30.432286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:30.526870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c9f394-255d-4fe0-8d4b-ba03dcc503c6 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:30.567304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:30.568202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:30.568612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:30.569041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:30.581836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:30.583556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:30.601669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:30.603874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:30.610893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:30.610893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:30.610893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:30.610893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 605/1215] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:30.611306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:30.684238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:30.731124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3fb1cf-279a-4888-aa7d-cc9e09611402 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:30.774512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:30.775174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:30.775449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:30.775719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:30.776439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:30.776739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:30.780641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:30.792978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 611/1216] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:30.799364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:30.882795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:30.919527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9cf149-ee6a-47ac-a412-70545960ddfe None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:30.960729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:30.961360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:30.961681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:30.962021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:30.970315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:30.972103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:30.986242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:30.987367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:30.989365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 606/1217] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:30.997063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:31.122226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5a26d-db3e-4cd2-8c9c-0c2987c519f7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:31.151999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:31.167254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:31.167881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.168186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.168464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.169175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:31.169478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:31.170953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab44463-0de2-43b9-a09e-71f071b0b33f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:31.185572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 612/1218] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:31.193038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:31.319345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780a01b1-6d3d-443a-85be-91bcdf1d8f1b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:31.354742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:31.362008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:31.362675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:31.363044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:31.363360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:31.364243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=59c91fec50c6477b8f7b7bc45534a2d5) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:31.364602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:31.366202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efc3071-c64c-4658-8121-5762fb253080 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:31.372588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 607/1219] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:31 2022] GET /identity/v3/role_assignments?user.id=59c91fec50c6477b8f7b7bc45534a2d5&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:33.654485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9c08-5694-4704-8e1d-964098a3da9d None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.655027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9c08-5694-4704-8e1d-964098a3da9d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.655169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0a9c08-5694-4704-8e1d-964098a3da9d None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.656745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 613/1220] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:33.663002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:33.663672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:33.663672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:33.685077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:33.686364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:33.800918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 608/1221] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:34.081155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a9c08-5694-4704-8e1d-964098a3da9d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:34.210793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0a9c08-5694-4704-8e1d-964098a3da9d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:34.253599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:34.254248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.254527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.254809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.266908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:34.272027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:34.273896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:34.275595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 614/1222] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:34.283153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:34.410904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b35f78-fb5d-4451-a64b-1a769729db89 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:34.440830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:34.454444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:34.455196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.459838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.459838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.459838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:34.459838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:34.459838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:34.469514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 609/1223] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:34.479219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:34.535729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:34.607211 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54d48a-7d98-4403-820d-63e6f5f1c932 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:34.647943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:34.648773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.649128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.649455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.666025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:34.666025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:34.683799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:34.685722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:34.687568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:34.687568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:34.687568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:34.687568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 615/1224] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:34.694539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:34.836826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc5a76c-96e8-4b29-ba4e-84b9933720b9 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:34.877587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:34.878279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:34.878609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:34.878933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:34.879672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:34.880027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:34.881610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:34.896994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 610/1225] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:34.903505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.020387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.028130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8346c102-5dbc-4a88-8f59-8b237ddff5b9 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.069084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.069801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.070130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.070451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.079457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.081000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:35.093938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.095656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:35.096806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 616/1226] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:35.101413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.156803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.218629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c746ce32-bda3-413a-82ea-e3309e1cd8a8 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.259745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.260402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.260681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.260959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.261727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:35.262034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.263511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.288860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 611/1227] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:35.291916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.418373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdedc54-8f80-4e20-9a18-c25b574ea5ce None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.460913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.461568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.461845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.462117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.474928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.476448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:35.492979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.495169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:35.499005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:35.499005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:35.499005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:35.499005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 617/1228] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:35.501465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.623546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658b601e-8b2a-48be-9a49-10e8039626a3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.626791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.673646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.674628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.675137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.675616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.676902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:35.677461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.680070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.702928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 612/1229] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:35.709574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:35.721156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:35.846906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35571fcb-c979-4767-9837-7ff46fdda829 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:35.895038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:35.895587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:35.895901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:35.896183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:35.903877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:35.906065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:35.919604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:35.922251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:35.924495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 618/1230] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:35.929611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.050446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5f305a-f356-4668-b01f-24e0842e9f06 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.095321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.095966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.096278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.096609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.097414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:36.097762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.099316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a7bfc8-8f00-49c5-8e39-6a1f33368fbc None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.114426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 613/1231] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:36.124902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:36.212176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:36.252740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e256948-f8bf-433b-9869-d54d9de33eac None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:36.291618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:36.292318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:36.292600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:36.292872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/59c91fec50c6477b8f7b7bc45534a2d5/roles/93106f0f0b244f67994be5290a6e956b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:36.311628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=59c91fec50c6477b8f7b7bc45534a2d5, role_id=93106f0f0b244f67994be5290a6e956b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:36.317229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d787f96-74b3-4258-bcc4-f341225f2a7e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:36.404660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 619/1232] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:36 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/59c91fec50c6477b8f7b7bc45534a2d5/roles/93106f0f0b244f67994be5290a6e956b => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:42:38.679540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1fcdd-e42d-4004-a140-b8dd40b6d325 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:38.680134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1fcdd-e42d-4004-a140-b8dd40b6d325 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:38.680134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d1fcdd-e42d-4004-a140-b8dd40b6d325 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:38.681789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 614/1233] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:38.687985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:38.688838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:38.688838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:38.708569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:38.709806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:38.798888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 620/1234] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:39.070835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1fcdd-e42d-4004-a140-b8dd40b6d325 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:39.118276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:39.229003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d1fcdd-e42d-4004-a140-b8dd40b6d325 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:39.298264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:39.299359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.299854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.300383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] PATH_INFO: `/v3/roles/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.323105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:39.332083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:39.333270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:39.334328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 615/1235] 10.222.0.97 () {56 vars in 1102 bytes} [Thu Sep 8 13:42:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:39.345374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:39.437278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:39.499360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48fead2-7a47-4b2a-9189-ba630e7f60ed None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:39.542901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:39.543696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.544110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.544533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.545379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] RBAC: Adding query filter params (name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:39.545776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:39.547367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:39.557675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 621/1236] 10.222.0.97 () {56 vars in 1103 bytes} [Thu Sep 8 13:42:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:39.568111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:39.708614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:39.726706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b2e24-1597-46dc-bf64-1a39c34e8c13 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:39.764285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:39.764924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.767010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.767010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.777588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:39.779973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:39.798523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:39.800427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:39.802235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:39.802235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:39.802235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:39.802235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 616/1237] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:39.807412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:39.933030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f80152-955f-4eb0-a0da-4637c3672767 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:39.944531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:39.975035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:39.975778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:39.976190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:39.976577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:39.977327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:39.977726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:39.979443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:39.994077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 622/1238] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:40.000426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.121871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03be3fbf-a814-405b-a27b-635c1eb3de6e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.163546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.164258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.164521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.164794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] PATH_INFO: `/v3/users/placement` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.168375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.174575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.176042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:40.189672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:40.196489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:40.197718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 617/1239] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:40.198730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.316710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e7d32-aa3c-4a02-b8d6-24f7dd8882e3 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.357811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.358426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.358699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.358967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.359793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:40.360095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.361695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.383076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 623/1240] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:40.390209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.529403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce74dd-9f6b-4f88-bfe1-5d3af9b1dd2d None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.576379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.576948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.577228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.577504 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.662680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.671136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.672696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.788170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 618/1241] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:40.794984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:40.826810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:40.917906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cc4764-3a19-44e4-82e3-56cd83b8af2c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:40.957276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:40.961082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:40.961082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:40.961082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:40.971234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:40.972995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:40.990748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:40.992560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:40.994375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:40.994375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:40.994375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:40.994375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 624/1242] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:41.000346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.113849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.121010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411cd6fc-604f-4545-ae61-2dacdeb7dac3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.180032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.183278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.183278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.183742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.185555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.186044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.188589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.210544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 619/1243] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:41.216650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.314091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:41.339952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57bf05-3bf5-4d14-8e88-6fed4be50033 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.381291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.381847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.382123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.382401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.397239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.400094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:41.425587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:41.436690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:41.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 625/1244] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:41.439211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.553178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5eba52-ea95-4df9-8b18-d2de7daa28b3 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.595923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.596573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.596849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.597118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.597834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.598131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.599577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4718da2c-ed30-4481-b380-e9721c74c7c2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.614344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 620/1245] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:41.620898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:41.745968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a66a46f-d77c-44bd-a6eb-d35c973199aa None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:41.785772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:41.786468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:41.786770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:41.787080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:41.787913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] RBAC: Adding query filter params (role.id=93106f0f0b244f67994be5290a6e956b, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=59c91fec50c6477b8f7b7bc45534a2d5) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:41.788319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:41.790014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8ed0a4-514f-4c86-a057-377c52681d55 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:41.797049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 626/1246] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:41 2022] GET /identity/v3/role_assignments?user.id=59c91fec50c6477b8f7b7bc45534a2d5&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=93106f0f0b244f67994be5290a6e956b => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:44.149525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010135-276e-4b1a-9f2e-b436887401e1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.150102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010135-276e-4b1a-9f2e-b436887401e1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.150102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0010135-276e-4b1a-9f2e-b436887401e1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.151458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 621/1247] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:44.157984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.158800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.158800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.165312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:44.179611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:44.181180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:44.260753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 627/1248] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:44.531968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010135-276e-4b1a-9f2e-b436887401e1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:44.553177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:44.656091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0010135-276e-4b1a-9f2e-b436887401e1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:44.703193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:44.703943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.704304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.704626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.717170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:44.722348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:44.724125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:44.725844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 622/1249] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:44.733854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:44.852517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e614af1-6401-4b4b-9f3c-623ecda053ce None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:44.888538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:44.893913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:44.894532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:44.894808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:44.895078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:44.895754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:44.896060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:44.897603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:44.908109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 628/1250] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:44.915270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.070151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.107052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed792480-9327-431c-8f32-e4b72fb45f99 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.172684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.173795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.174325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.174815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.194501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.197210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:45.227126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:45.228907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:45.239458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:45.239458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:45.239458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:45.239458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 623/1251] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:45.239713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.358138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c402eb3-1a6d-4003-a5db-689f13d3e6a6 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.402064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.402859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.403151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.403418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.404093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:45.404435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.405948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.420771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 629/1252] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:45.427235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.553180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfe6d30-c39a-4961-8006-ca40e2da1372 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.586432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.595764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.596452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.596733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.597007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] PATH_INFO: `/v3/users/placement` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.606578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.608066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:45.630215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:45.631395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:45.632831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 624/1253] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/users/placement => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:45.638451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.709407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:45.762386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c07438-6a6b-4649-a825-c2a7eab4f4a1 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:45.802503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707455-5179-4fa5-823c-56654011423f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:45.803128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707455-5179-4fa5-823c-56654011423f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:45.803411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707455-5179-4fa5-823c-56654011423f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:45.803692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5707455-5179-4fa5-823c-56654011423f None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:45.804742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5707455-5179-4fa5-823c-56654011423f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:45.805088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5707455-5179-4fa5-823c-56654011423f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:45.806581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5707455-5179-4fa5-823c-56654011423f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:45.829112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 630/1254] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:45.837756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:45.960730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aca1aa-a100-413c-8400-e5fad2f1920e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.000480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.001105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.001385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.001656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.102242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.125406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.128280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.327217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 625/1255] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 08 13:42:46.333408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707455-5179-4fa5-823c-56654011423f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.458135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5707455-5179-4fa5-823c-56654011423f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.490655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.516409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.517057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.517336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.517616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.530935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.532631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:46.549371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:46.551165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:46.552431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:46.552431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:46.552431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:46.552431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 631/1256] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:46.559517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.663864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:46.677627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7512e55-595c-4ba6-a8aa-ad6c4ad5be1b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.717382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.718079 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.718358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.718634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.719348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:46.719667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.721772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.736434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 626/1257] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:46.742736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:46.873028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adca7f2-1357-4ee5-807c-18e374eedda0 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:46.915314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:46.915948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:46.916333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:46.916656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:46.925541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:46.927072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:46.941705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:46.943388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:46.944631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 632/1258] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:46.951492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:47.114129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0456e52-c7ca-465f-8044-de4d9c81f0b2 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:47.156119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:47.156932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:47.157324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:47.157644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:47.158553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:47.158976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:47.160700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f262796d-aafd-48e2-9175-9aedbab5aefe None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:47.176276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 627/1259] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:47.183551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:47.213567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:47.301063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e934ee4-090c-4752-968e-25e3777b700e None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:47.349660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:47.350646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:47.351105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:47.351559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:47.352874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=59c91fec50c6477b8f7b7bc45534a2d5) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:47.353390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:47.355892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22316ea5-371f-4b17-bef7-4de079439bfe None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:47.366039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 633/1260] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:47 2022] GET /identity/v3/role_assignments?user.id=59c91fec50c6477b8f7b7bc45534a2d5&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:49.638938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d793-d7b1-4ffb-baac-4506614a3f2a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:49.639509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d793-d7b1-4ffb-baac-4506614a3f2a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:49.639849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5d793-d7b1-4ffb-baac-4506614a3f2a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:49.642191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 628/1261] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:42:49.648451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:49.649044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:49.649208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:49.670058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:49.674096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:49.751700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 634/1262] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:42:50.025147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d793-d7b1-4ffb-baac-4506614a3f2a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.037903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.143337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5d793-d7b1-4ffb-baac-4506614a3f2a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.186631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.187269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.187545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.187811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.199577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:50.204641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:50.210435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:50.211286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 629/1263] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:50.213238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.234450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.345187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa63746-423b-486d-9d25-b735e631cbf0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.411137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.412231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.412734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.413199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.414418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] RBAC: Adding query filter params (name=admin) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:50.414955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.417724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.434946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 635/1264] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:50.443048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.529901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.571053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f411bbc-7729-4cb3-a4e2-24496b4d19d4 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.616445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.617284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.617619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.617983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.631791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.633484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:50.650345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:50.654087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:50.655091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:50.655091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:50.655091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:50.655091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 630/1265] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:50.658336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.754486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:50.777006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40de894-89a1-4ab8-9ef0-96f8610c085f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:50.815603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:50.816257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:50.816537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:50.816812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:50.817480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:50.817790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:50.819180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:50.833677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 636/1266] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:50.841566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:50.991619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0664246c-ed1c-4526-b56d-9712bca7a6cf None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.015395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.034028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.034672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.034953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.035226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] PATH_INFO: `/v3/users/placement` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.044756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.046328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:51.059867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:51.061825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:51.063720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 631/1267] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:51.070851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.191738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed939473-1e82-4d22-b392-5f25dd95881c None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.235728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.236394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.236674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.236946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.237689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:51.238000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.239492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.250078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.260980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 637/1268] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:51.267408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.384520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d960943-aae5-419e-a64f-5a28b1665305 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.430485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.431271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.431645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.432010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.446100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.447488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:51.464133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:51.466082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:51.467597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:51.467597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:51.467597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:51.467597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 632/1269] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:51.471488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.595936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84829139-ca4e-4da4-99a1-808f9641299f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.637862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.638682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.639056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.639421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.640242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:51.640649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.642302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.656221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 638/1270] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:51.662164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.702643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.782207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd910e4e-c1f1-4029-b7aa-1ccf1c80259e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:51.822735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:51.823426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:51.823736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:51.824014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:51.831791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:51.833423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:51.847272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:51.849361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:51.851688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 633/1271] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:51.857846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:51.927775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:51.975518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4991dfe4-fb54-4bf2-86eb-61bb2fc91f07 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:52.034591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:52.035628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:52.036126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:52.036581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:52.037717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:52.038231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:52.040839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94da1736-1ec5-4037-a953-891e318904f0 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:52.062697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 639/1272] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:52.073536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:52.183706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:52.194599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331543de-7ad0-4cf0-9268-5ecf5aeada1b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:52.242477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:52.243109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:52.243446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:52.243815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] PATH_INFO: `/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/59c91fec50c6477b8f7b7bc45534a2d5/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:52.246196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] RBAC: Authorizing `identity:create_grant(project_id=5a3348fedcb34424aba1f8f7828968cb, user_id=59c91fec50c6477b8f7b7bc45534a2d5, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:52.252348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3570dd3-438d-4206-8ffc-abf30e49b634 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:52.470532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 634/1273] 10.222.0.97 () {58 vars in 1439 bytes} [Thu Sep 8 13:42:52 2022] PUT /identity/v3/projects/5a3348fedcb34424aba1f8f7828968cb/users/59c91fec50c6477b8f7b7bc45534a2d5/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:42:54.764094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ac53-f511-4123-aec2-13e93fa8fd8f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.764753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ac53-f511-4123-aec2-13e93fa8fd8f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.764753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1ac53-f511-4123-aec2-13e93fa8fd8f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.766072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 640/1274] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:42:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:42:54.775932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:54.776544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:54.776544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:54.796757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:42:54.797971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:42:54.886326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 635/1275] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:42:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:42:55.239722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1ac53-f511-4123-aec2-13e93fa8fd8f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.373455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1ac53-f511-4123-aec2-13e93fa8fd8f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.399968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.418637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.419342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.419626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.419897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.433455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 08 13:42:55.438516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:55.440367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:55.442048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 641/1276] 10.222.0.97 () {56 vars in 1096 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:55.448652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.550869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:55.580322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52bf352-6cf7-415c-a0bf-038cdb3cf0a4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.642896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.643738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.644168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.644648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.645757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] RBAC: Adding query filter params (name=admin) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:55.646273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.648926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.661609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 636/1277] 10.222.0.97 () {56 vars in 1099 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:55.669083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:55.791471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e40b42-5d78-4150-9647-bdff8706203f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:55.831986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:55.832677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:55.833035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:55.833351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:55.846576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:55.849601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:55.875434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:55.878085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:55.879875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:55.879875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:55.879875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:55.879875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 642/1278] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:42:55.887087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:56.067039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6531884-79a4-43c3-900d-4210e7322c7c None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:56.105719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:56.106273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:56.106552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:56.106825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:56.107487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:56.107789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:56.109214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:56.123222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 637/1279] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:56.129413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:56.166477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:56.245386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bd947-9523-4831-99af-acd4cf5e1bd7 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:56.287273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:56.287963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:56.288389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:56.288787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] PATH_INFO: `/v3/users/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:56.298467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:56.299939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 08 13:42:56.313335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:56.315047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:42:56.316932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 643/1280] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:56.323054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:56.327402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:56.449247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc30b7-9335-487b-89f6-953e8625b708 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:56.487679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:56.488331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:56.488614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:56.488878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:56.489619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:56.489918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:56.491328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:56.514027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 638/1281] 10.222.0.97 () {56 vars in 1143 bytes} [Thu Sep 8 13:42:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:56.520632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:56.637340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:56.649782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86e1738-aa67-49c3-8b29-dc7b2c1dd33f None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:56.694718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:56.695406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:56.695727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:56.696055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:56.794447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:56.797078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:56.952493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:56.973997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 644/1282] 10.222.0.97 () {58 vars in 1282 bytes} [Thu Sep 8 13:42:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 08 13:42:56.981388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:57.105673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:57.181907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d33c2ca-c298-49ca-aede-597387752ebd None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:57.226587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:57.227228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:57.227507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:57.228544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:57.240128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:57.241617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 08 13:42:57.257693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:57.271843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 08 13:42:57.273641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 08 13:42:57.273641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 08 13:42:57.273641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:57.273641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 639/1283] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:42:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:57.274074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:57.396426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace1c742-34ba-4267-abdd-2bf17215b688 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:57.438117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:57.443447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:57.454857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 645/1284] 10.222.0.97 () {56 vars in 1109 bytes} [Thu Sep 8 13:42:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:42:57.463944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:57.621727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01142bda-c1b8-4b99-9644-984b3296c05b None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:57.637934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:57.661389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:57.661955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:57.662235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:57.662510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] PATH_INFO: `/v3/projects/service` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:57.670941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:57.672530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 08 13:42:57.685560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application with Lock( Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:42:57.686797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 08 13:42:57.687831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: ERROR keystone.server.flask.application Sep 08 13:42:57.695500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 640/1285] 10.222.0.97 () {56 vars in 1111 bytes} [Thu Sep 8 13:42:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:42:57.695647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:57.753715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:42:57.815022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e2713d-d663-4b24-9550-d0a11fe3de8d None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:57.856751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:57.857787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:57.858337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:57.858822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:57.860373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:57.860902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:57.863586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416cc92c-1190-4d1c-b343-cc163be510ef None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:57.889141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 646/1286] 10.222.0.97 () {56 vars in 1148 bytes} [Thu Sep 8 13:42:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:42:57.896074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:42:58.014021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddd580-1654-41fe-8a89-ff80f1a9761e None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:42:58.053734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:42:58.054357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:42:58.054635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:42:58.054908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:42:58.055660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] RBAC: Adding query filter params (role.id=e7c0471ae3484d85b6e31650411b03e6, scope.project.id=5a3348fedcb34424aba1f8f7828968cb, user.id=59c91fec50c6477b8f7b7bc45534a2d5) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:42:58.055964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:42:58.057459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614971e3-de52-4ce2-b93c-7a47c55d0771 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:42:58.064411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 641/1287] 10.222.0.97 () {56 vars in 1374 bytes} [Thu Sep 8 13:42:57 2022] GET /identity/v3/role_assignments?user.id=59c91fec50c6477b8f7b7bc45534a2d5&scope.project.id=5a3348fedcb34424aba1f8f7828968cb&role.id=e7c0471ae3484d85b6e31650411b03e6 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:43:00.392186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596ef1b-46ce-4442-b121-c4669a0ec188 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.392862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596ef1b-46ce-4442-b121-c4669a0ec188 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.392862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596ef1b-46ce-4442-b121-c4669a0ec188 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.394100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 647/1288] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:43:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:00.400875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.401481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.401481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.420792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:00.422066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:00.455456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:00.501437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 642/1289] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:00.775201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596ef1b-46ce-4442-b121-c4669a0ec188 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:00.903330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596ef1b-46ce-4442-b121-c4669a0ec188 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:00.916832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:00.943007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:00.943660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:00.943939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:00.944249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:00.944859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:00.946286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:00.956314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:00.957642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:00.958949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 648/1290] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:00 2022] GET /identity/v3/services/placement => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:00.964962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:01.076426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6da39-0796-4ca7-a8e8-1c627aecb7c7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:01.113262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:01.116120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:01.116120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:01.116120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:01.116120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] RBAC: Adding query filter params (name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:01.116120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:01.116681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1393111e-3cad-4223-9c77-f838f0b30f5b None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:01.122435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 643/1291] 10.222.0.97 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:00 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:01.128203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:01.247412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90abe2ac-9673-4d7b-aa27-e2e9b1b3b550 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:01.289118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:01.289943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:01.290303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:01.290635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] PATH_INFO: `/v3/services` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:01.291333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] RBAC: Adding query filter params (type=placement) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:01.291710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:01.293417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d526982e-b818-4a88-85ba-3e0f1fcf8d4d None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:01.299472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 649/1292] 10.222.0.97 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:01 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:03.592539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a04-871f-46bc-9655-0df6ebe2ebf1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:03.593170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a04-871f-46bc-9655-0df6ebe2ebf1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:03.593170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d9a04-871f-46bc-9655-0df6ebe2ebf1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:03.594440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 644/1293] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:43:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:03.602598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be19-e5d6-499b-94f0-c139e9f852f6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:03.603199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be19-e5d6-499b-94f0-c139e9f852f6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:03.603522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3be19-e5d6-499b-94f0-c139e9f852f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:03.628875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-04a3be19-e5d6-499b-94f0-c139e9f852f6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:03.630179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-04a3be19-e5d6-499b-94f0-c139e9f852f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:03.668042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:03.712975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 650/1294] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:03.986273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d9a04-871f-46bc-9655-0df6ebe2ebf1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:04.020904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:04.103450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d9a04-871f-46bc-9655-0df6ebe2ebf1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:04.141575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:04.142193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:04.142468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:04.142736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:04.143404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:04.144826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d6572f-09ac-43b9-ae15-2eaeea0b8b29 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:04.255663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 645/1295] 10.222.0.97 () {60 vars in 1141 bytes} [Thu Sep 8 13:43:03 2022] POST /identity/v3/services => generated 246 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:43:06.573719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486e5e8-136d-4d11-bad4-5117c531db80 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:06.574322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486e5e8-136d-4d11-bad4-5117c531db80 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:06.574322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486e5e8-136d-4d11-bad4-5117c531db80 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:06.575599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 651/1296] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:43:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:06.581730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:06.583272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:06.583272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:06.604389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:06.605643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:06.693281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 646/1297] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:06 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 08 13:43:06.971482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486e5e8-136d-4d11-bad4-5117c531db80 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:07.091443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486e5e8-136d-4d11-bad4-5117c531db80 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:07.131509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:07.132703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:07.132980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:07.133248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:07.133836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:07.135238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:07.145021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:07.148938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:07.149877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 652/1298] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:06 2022] GET /identity/v3/services/placement => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 08 13:43:07.157189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:07.188920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:07.282573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88979210-51a1-463a-8a42-93e2418c4042 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:07.323617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:07.324336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:07.324662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:07.325000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:07.325738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:07.326092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:07.327521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:07.333640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 647/1299] 10.222.0.97 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:07 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:07.345003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:07.406308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:07.510022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae7561-0073-4977-88f1-fa4d9ddc55bc None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:07.549196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:07.549721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:07.550264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:07.550530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:07.551262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] RBAC: Adding query filter params (interface=public, service_id=99a8da6ff09a4be28a7aceb0810a9b50, region_id=RegionOne) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:07.551574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:07.553018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef29bced-5467-471a-9068-08a8a787d655 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:07.560904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 653/1300] 10.222.0.97 () {56 vars in 1251 bytes} [Thu Sep 8 13:43:07 2022] GET /identity/v3/endpoints?service_id=99a8da6ff09a4be28a7aceb0810a9b50&interface=public®ion_id=RegionOne => generated 194 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:07.567217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:07.667333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:07.688427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6111b3b-a24f-417c-b778-1bda4275e3a7 None admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:07.728636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:07.729263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:07.729545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:07.729821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:07.730433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:07.731808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688eed3e-fb42-4124-b11a-aeb9772aaddf None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:07.739968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 648/1301] 10.222.0.97 () {56 vars in 1087 bytes} [Thu Sep 8 13:43:07 2022] GET /identity/v3/services => generated 2191 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:43:10.080563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcef38-56e2-44e3-a6b5-31a3ff89c26a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:10.081217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcef38-56e2-44e3-a6b5-31a3ff89c26a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:10.081217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dcef38-56e2-44e3-a6b5-31a3ff89c26a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:10.082615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 654/1302] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:43:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:43:10.089538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:10.090175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:10.090175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:10.110746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:43:10.112061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:43:10.194210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 649/1303] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:43:10 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 08 13:43:10.517953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcef38-56e2-44e3-a6b5-31a3ff89c26a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:10.649482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:10.657213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dcef38-56e2-44e3-a6b5-31a3ff89c26a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:10.698578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:10.699217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:10.699492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:10.699763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] PATH_INFO: `/v3/services/placement` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:10.700401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:10.701792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 08 13:43:10.712419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 08 13:43:10.714348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.get_or_create( Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application with Lock( Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self._enter() Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application return self.creator() Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application created_value = creator( Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: ERROR keystone.server.flask.application Sep 08 13:43:10.716359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 655/1304] 10.222.0.97 () {56 vars in 1117 bytes} [Thu Sep 8 13:43:10 2022] GET /identity/v3/services/placement => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 08 13:43:10.723467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:10.813899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:10.839117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a99a09-1b42-4a48-91bd-58fe6f2edf7e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:10.878937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:10.879541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:10.879823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:10.880089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] PATH_INFO: `/v3/services` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:10.880898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] RBAC: Adding query filter params (name=placement) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:43:10.881203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:10.882572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6552c7-8319-4c73-ad20-372ca8c5208d None admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:10.890675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 650/1305] 10.222.0.97 () {56 vars in 1116 bytes} [Thu Sep 8 13:43:10 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 13:43:10.897783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:43:11.020386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a9478-cbe0-4257-87c6-88b46db6d624 None admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:43:11.066988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:43:11.068052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:43:11.069188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:43:11.069678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:43:11.070959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:43:11.073793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ffc84f-f823-4ce0-a29f-5db114f62fb7 None admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:43:11.169266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 656/1306] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:43:10 2022] POST /identity/v3/endpoints => generated 331 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:43:20.888048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=30 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:43:21.163885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:26.990356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d410a-d24f-4c19-8d9a-1f03ff5c0b31 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:26.991177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d410a-d24f-4c19-8d9a-1f03ff5c0b31 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:26.991177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d410a-d24f-4c19-8d9a-1f03ff5c0b31 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:26.998604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 651/1307] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:27.001968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a98e-86ce-4813-88a8-b58a0c033084 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.003384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a98e-86ce-4813-88a8-b58a0c033084 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.004010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f61a98e-86ce-4813-88a8-b58a0c033084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:27.151739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8f61a98e-86ce-4813-88a8-b58a0c033084 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:27.154391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8f61a98e-86ce-4813-88a8-b58a0c033084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:27.359023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 657/1308] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:27.922355 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.923155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.928400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:27.928400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 652/1309] 10.222.0.97 () {54 vars in 965 bytes} [Thu Sep 8 13:52:27 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:27.937044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9494-12f0-4a7e-bfab-f0eac161b08d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:27.937484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9494-12f0-4a7e-bfab-f0eac161b08d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:27.937955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06f9494-12f0-4a7e-bfab-f0eac161b08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.083478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e06f9494-12f0-4a7e-bfab-f0eac161b08d None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:28.085787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e06f9494-12f0-4a7e-bfab-f0eac161b08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.115461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:28.185000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 658/1310] 10.222.0.97 () {58 vars in 1062 bytes} [Thu Sep 8 13:52:27 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:28.204101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:28.213366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.499011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b495a8c-2404-4250-865a-2cc2898ac584 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:28.653739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:28.654454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:28.654772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:28.655176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:28.657196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:28.762303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:28.840724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:28.842176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecab4912-f6c4-4a7d-8bef-d2b6a25a843c service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:28.872080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 653/1311] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:31.576649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.577510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.577510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.579234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 659/1312] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:31.585315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02687-485f-4ec7-91ab-e78c61476f8a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:31.585881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02687-485f-4ec7-91ab-e78c61476f8a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:31.585881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da02687-485f-4ec7-91ab-e78c61476f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:31.685825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1da02687-485f-4ec7-91ab-e78c61476f8a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:31.687143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1da02687-485f-4ec7-91ab-e78c61476f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:31.801565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 654/1313] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:32.095974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:32.097665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:32.224355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0d4d96-ff2d-4970-849c-fa7a350a2e33 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:32.336025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:32.340687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:32.341656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:32.342144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:32.342618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:32.346038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:32.559403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:32.562289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46f9700-3390-44ac-b1ae-7e68b28df5b8 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:32.592487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 660/1314] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:35.610614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.611365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.611365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.612499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 655/1315] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:35.619760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b883368-cbde-4ac2-a285-e0b5dc44bf6f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:35.620984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b883368-cbde-4ac2-a285-e0b5dc44bf6f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:35.621351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b883368-cbde-4ac2-a285-e0b5dc44bf6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:35.698884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1b883368-cbde-4ac2-a285-e0b5dc44bf6f None None] MFA Rules not processed for user `b9aec71002b149c6b1beb79cdf0db364`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:35.700092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1b883368-cbde-4ac2-a285-e0b5dc44bf6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:35.762088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:35.815434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 661/1316] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:35 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:36.109840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:36.112290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.124897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.239871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e70720d-8721-4b8b-aeb0-4ec7d16448f1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:36.355577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:36.356111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:36.356441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:36.356700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:36.358465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:36.434982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:36.489158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:36.490662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2572cb-7d71-4fcf-8c4e-ce5064db52f0 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:36.512454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 656/1317] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:36 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:40.133303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe849-0707-47f4-8010-f158f5b24d4b None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.133867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe849-0707-47f4-8010-f158f5b24d4b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.133867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bfe849-0707-47f4-8010-f158f5b24d4b None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.135189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 662/1318] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:40.141627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045a635-2bfe-4da1-a883-60916004c889 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.142122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045a635-2bfe-4da1-a883-60916004c889 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.142239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5045a635-2bfe-4da1-a883-60916004c889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.217709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5045a635-2bfe-4da1-a883-60916004c889 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:40.218972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5045a635-2bfe-4da1-a883-60916004c889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.337388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 657/1319] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:40.652464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.652849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.653245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.655155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 663/1320] 10.222.0.97 () {54 vars in 965 bytes} [Thu Sep 8 13:52:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:40.662370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849a61-643a-4c69-940a-8a94ad368eee None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:40.662730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849a61-643a-4c69-940a-8a94ad368eee None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:40.663077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a849a61-643a-4c69-940a-8a94ad368eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:40.695125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:40.760360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9a849a61-643a-4c69-940a-8a94ad368eee None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:40.762194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9a849a61-643a-4c69-940a-8a94ad368eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.851742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 658/1321] 10.222.0.97 () {58 vars in 1062 bytes} [Thu Sep 8 13:52:40 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:52:40.862280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:40.863955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:40.977702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7defc61-16ff-4ab6-856d-a5c9e5684e58 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:41.059241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:41.059898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:41.060200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:41.060469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:41.062153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:41.070076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:41.185791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:41.187141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0a99d6-05a0-47c1-8954-db3361aa8eac service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:41.208081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 664/1322] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:40 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:45.227547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.228133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.228364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.229923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 659/1323] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:45.236398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2acb-1f9e-4464-b1b7-ff1b9a235fd6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.237326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2acb-1f9e-4464-b1b7-ff1b9a235fd6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.237625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7a2acb-1f9e-4464-b1b7-ff1b9a235fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.316807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3c7a2acb-1f9e-4464-b1b7-ff1b9a235fd6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:45.318092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3c7a2acb-1f9e-4464-b1b7-ff1b9a235fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.385956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.433886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 665/1324] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:45.734791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:45.736769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:45.863421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8397494-63ed-4a0a-9c65-4cd7acd4f6b9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:45.882184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:45.985288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:45.985800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:45.986072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:45.986338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:45.988072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:46.145496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:46.147197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555216a-237d-4737-9ea7-75400dc7ac75 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:46.170370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 660/1325] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:48.605378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269449-924f-44df-8067-386963e87698 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.605902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269449-924f-44df-8067-386963e87698 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.605902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac269449-924f-44df-8067-386963e87698 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:48.607377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 666/1326] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:48.614285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0f4f-01f2-4d6b-b6fe-697492a0918b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:48.615137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0f4f-01f2-4d6b-b6fe-697492a0918b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:48.615327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1d0f4f-01f2-4d6b-b6fe-697492a0918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:48.684844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:48.710786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-be1d0f4f-01f2-4d6b-b6fe-697492a0918b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:48.712113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-be1d0f4f-01f2-4d6b-b6fe-697492a0918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:48.849376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 661/1327] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:49.382282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269449-924f-44df-8067-386963e87698 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:49.384084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ac269449-924f-44df-8067-386963e87698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.554686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac269449-924f-44df-8067-386963e87698 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:49.684944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:49.685820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:49.686278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:49.686733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:49.690051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:49.724093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:49.892196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:49.895276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877c0dd-f779-4687-9d82-91209671e77b service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:49.933247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 667/1328] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 552 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:52:52.380031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.380612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.381077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.382816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 662/1329] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:52:52.393091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb14d-e863-486f-922f-66a7fdd5d5b9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:52.393604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb14d-e863-486f-922f-66a7fdd5d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:52.394064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdcb14d-e863-486f-922f-66a7fdd5d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:52.493495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-fbdcb14d-e863-486f-922f-66a7fdd5d5b9 None None] MFA Rules not processed for user `b9aec71002b149c6b1beb79cdf0db364`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:52.495497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fbdcb14d-e863-486f-922f-66a7fdd5d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:52.577708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:52.630031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 668/1330] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:52.927978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:52.929793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.077387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fec9b6-4bf4-43dd-b80e-87e658a16aef None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:53.102032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:53.217921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:53.218993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:53.219489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:53.219990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:53.223326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:53.434147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:53.436945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc58a37a-729d-48ce-9a3e-fa4c38821f52 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:53.474034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 663/1331] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:52 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 546 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:56.512418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.513304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.513304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.514333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 669/1332] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:56.520776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e547-8675-427a-bf59-ad8d05eb9899 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:56.521342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e547-8675-427a-bf59-ad8d05eb9899 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:56.521342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d1e547-8675-427a-bf59-ad8d05eb9899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:56.622402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:56.626029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-17d1e547-8675-427a-bf59-ad8d05eb9899 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:52:56.627361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-17d1e547-8675-427a-bf59-ad8d05eb9899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:56.743901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 664/1333] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:52:57.032548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:52:57.034185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:57.159850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89816ee3-2b92-4f17-ba8d-b5faabe7a950 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:52:57.247463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:52:57.248079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:57.248391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:57.248670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:57.250396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:52:57.283452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:52:57.447288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:52:57.449668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e0cfbc-14c0-4320-97c1-e4a693773dc7 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:52:57.487035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 670/1334] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:52:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:52:59.962851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.963432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.963620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:52:59.965653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 665/1335] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:52:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:52:59.972063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29593-d936-41bc-8aad-91cb3b1a77cf None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:52:59.979938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29593-d936-41bc-8aad-91cb3b1a77cf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:52:59.979938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d29593-d936-41bc-8aad-91cb3b1a77cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:00.054428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-19d29593-d936-41bc-8aad-91cb3b1a77cf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:00.055665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-19d29593-d936-41bc-8aad-91cb3b1a77cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:00.145778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:00.172840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 671/1336] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:00.473264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:00.476070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:00.613283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb03558e-0f31-46a6-83d6-24e046ed8ac2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:00.648081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:00.734269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:00.735336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:00.735855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:00.736362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:00.739458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:00.953819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:00.956533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28e006e-5ffd-4d55-bd6e-a371eaf17f44 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:00.984004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 666/1337] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:04.267068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:04.267645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:04.267774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:04.269438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 672/1338] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:04.277448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd370dc-7c7d-4176-8742-79e41ddb4090 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:04.277997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd370dc-7c7d-4176-8742-79e41ddb4090 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:04.278332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd370dc-7c7d-4176-8742-79e41ddb4090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:04.385256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8bd370dc-7c7d-4176-8742-79e41ddb4090 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:04.387115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8bd370dc-7c7d-4176-8742-79e41ddb4090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:04.397385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:04.501354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 667/1339] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:04.790992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:04.792706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:04.915561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f03ed61-5731-4072-b3c7-53ef1d1e715f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:05.009272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:05.009883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:05.010162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:05.010426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:05.012319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:05.055937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:05.149160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:05.150653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe037d5-e6b9-4c88-a4bf-4ed826f1d8f4 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:05.173586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 673/1340] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:10.643918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:10.644793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:10.644793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:10.645933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 668/1341] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:53:10.653061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6af54-7576-44ec-b13c-6db79ad6d094 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:10.653855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6af54-7576-44ec-b13c-6db79ad6d094 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:10.654018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca6af54-7576-44ec-b13c-6db79ad6d094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:10.752093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0ca6af54-7576-44ec-b13c-6db79ad6d094 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:10.753325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0ca6af54-7576-44ec-b13c-6db79ad6d094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:10.856270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:10.868586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 674/1342] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:11.157296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:11.158977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.294802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8fae42-bba9-4b98-8d0f-6411acb1a8bf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:11.350927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:11.394024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:11.394664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:11.394943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:11.395216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:11.396927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:11.536759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:11.538222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ee793e-a0a6-498a-a553-7ade82a3a2f1 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:11.560129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 669/1343] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:15.237177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.237987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.238168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.239666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 675/1344] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:15.246544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917d53d-bdad-4b0f-9819-ff8312e2f567 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:15.247388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917d53d-bdad-4b0f-9819-ff8312e2f567 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:15.247388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f917d53d-bdad-4b0f-9819-ff8312e2f567 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:15.363818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f917d53d-bdad-4b0f-9819-ff8312e2f567 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:15.365175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f917d53d-bdad-4b0f-9819-ff8312e2f567 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.388103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:15.537983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 670/1345] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:15 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:15.834078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:15.835812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:15.974303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ffbfa-7271-46c6-a698-04ce87fee8f9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:16.077874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:16.077874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:16.077874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:16.077874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:16.078436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:16.133293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:16.210186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:16.211655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ffeb5e1-55f9-4bf6-b1e6-2d782bcab327 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:16.233469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 676/1346] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:15 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:21.323199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.323903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.323903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.325243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 671/1347] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:21.332224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb79348-03a9-429f-b5d3-37a0061f5411 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:21.332803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb79348-03a9-429f-b5d3-37a0061f5411 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:21.333187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb79348-03a9-429f-b5d3-37a0061f5411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:21.425134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4eb79348-03a9-429f-b5d3-37a0061f5411 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:21.426543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4eb79348-03a9-429f-b5d3-37a0061f5411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:21.594931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:21.598035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 677/1348] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:21.934388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:21.936260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.058359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd462ee-fd16-4131-94bb-f2f5c3dba576 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:22.123322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:22.161867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:22.162499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:22.162859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:22.163152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:22.167338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:22.333185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:22.334692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b8e36f-72cc-459a-8730-28b2df52eefd service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:22.356446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 672/1349] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:28.122546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d9ecf-7bd0-4165-af39-a91c0812bf28 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.123288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d9ecf-7bd0-4165-af39-a91c0812bf28 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.123525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d9ecf-7bd0-4165-af39-a91c0812bf28 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.126627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 678/1350] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 13:53:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:28.137977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c749-4cd9-4840-8467-01e98e8eb4c3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:28.139191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c749-4cd9-4840-8467-01e98e8eb4c3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:28.139529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103c749-4cd9-4840-8467-01e98e8eb4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:28.272828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5103c749-4cd9-4840-8467-01e98e8eb4c3 None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:28.276312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5103c749-4cd9-4840-8467-01e98e8eb4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:28.321663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:28.505904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 673/1351] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:28 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:33.996899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f87f18-5432-46a3-b491-9f0f2bda2073 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:33.997945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f87f18-5432-46a3-b491-9f0f2bda2073 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:33.998569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f87f18-5432-46a3-b491-9f0f2bda2073 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.000176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 679/1352] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 13:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:34.006392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5d52-b40b-4d46-8926-aa70f7e26338 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:34.006987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5d52-b40b-4d46-8926-aa70f7e26338 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:34.006987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945c5d52-b40b-4d46-8926-aa70f7e26338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:34.241559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-945c5d52-b40b-4d46-8926-aa70f7e26338 None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:34.243242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-945c5d52-b40b-4d46-8926-aa70f7e26338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:34.348752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:34.392213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 674/1353] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:34 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:35.545492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe431-882e-4b9e-b622-119a9da6043f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:35.546348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe431-882e-4b9e-b622-119a9da6043f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:35.546348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655fe431-882e-4b9e-b622-119a9da6043f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:35.548171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 680/1354] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 13:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:35.555375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307b9a2-7229-4d76-9e91-2c0601d12d69 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:35.556004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307b9a2-7229-4d76-9e91-2c0601d12d69 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:35.556641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3307b9a2-7229-4d76-9e91-2c0601d12d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:35.687655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3307b9a2-7229-4d76-9e91-2c0601d12d69 None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:35.690025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3307b9a2-7229-4d76-9e91-2c0601d12d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:35.866453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 675/1355] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 13:53:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:40.573184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a15bb-2497-4af5-8fab-1834bcd84697 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.573986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a15bb-2497-4af5-8fab-1834bcd84697 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.573986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68a15bb-2497-4af5-8fab-1834bcd84697 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.576312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 681/1356] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:40.584754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd812-a54e-40af-ac01-4b1b995f43b1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:40.585516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd812-a54e-40af-ac01-4b1b995f43b1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:40.585516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66dd812-a54e-40af-ac01-4b1b995f43b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:40.650476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:40.717373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f66dd812-a54e-40af-ac01-4b1b995f43b1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:40.718742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f66dd812-a54e-40af-ac01-4b1b995f43b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:40.949908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 676/1357] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:41.278218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.278706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.279195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.283717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 682/1358] 10.222.0.97 () {54 vars in 962 bytes} [Thu Sep 8 13:53:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:41.300238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7fa81-7ad7-4f1f-959f-ac9a574d6e11 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:41.300706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7fa81-7ad7-4f1f-959f-ac9a574d6e11 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:41.301259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f7fa81-7ad7-4f1f-959f-ac9a574d6e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:41.460842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-47f7fa81-7ad7-4f1f-959f-ac9a574d6e11 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:41.462823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-47f7fa81-7ad7-4f1f-959f-ac9a574d6e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.539239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:41.644578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 677/1359] 10.222.0.97 () {58 vars in 1059 bytes} [Thu Sep 8 13:53:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:41.658961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:41.660902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:41.846729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a806b0-169b-4aa6-98a4-b3c7d93f797f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:41.978214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:42.065540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:42.066456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:42.066892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:42.067460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:42.070430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:42.265451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:42.268554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deddb06-9e82-444e-8a4f-e315a26e43c3 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:42.309000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 683/1360] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 650 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:45.107293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.107992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.109154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.113146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 678/1361] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:45 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:45.119148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d53e-6ead-4cc9-bf47-123c10f320e4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:45.119148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d53e-6ead-4cc9-bf47-123c10f320e4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:45.121235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d35d53e-6ead-4cc9-bf47-123c10f320e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:45.216415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6d35d53e-6ead-4cc9-bf47-123c10f320e4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:45.218402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6d35d53e-6ead-4cc9-bf47-123c10f320e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:45.293433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:45.431574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 684/1362] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:45.966505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:45.970316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.202103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de99ef5-89cb-4de9-aea8-ecb3b10b3591 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:46.213126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:46.372555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:46.373487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:46.373765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:46.374112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:46.375908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:46.497546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:46.499574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f081bf-a9fb-4a02-9e2f-b48e96641a94 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:46.525260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 679/1363] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:49.606152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef7fb-c5b0-4360-bfc1-c40a0e57c85e None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.607029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef7fb-c5b0-4360-bfc1-c40a0e57c85e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.607465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5ef7fb-c5b0-4360-bfc1-c40a0e57c85e None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.613677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 685/1364] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:49 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:49.616612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbde986-c2e5-4486-9889-25aa728d3471 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:49.617271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbde986-c2e5-4486-9889-25aa728d3471 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:49.617891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbde986-c2e5-4486-9889-25aa728d3471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:49.652472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:49.749375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0fbde986-c2e5-4486-9889-25aa728d3471 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:49.751488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0fbde986-c2e5-4486-9889-25aa728d3471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:49.949199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 680/1365] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:50.300936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf53e-4726-4a0b-b7b4-4f066ba29f42 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.301443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf53e-4726-4a0b-b7b4-4f066ba29f42 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.301929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ebf53e-4726-4a0b-b7b4-4f066ba29f42 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.304528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 686/1366] 10.222.0.97 () {54 vars in 962 bytes} [Thu Sep 8 13:53:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:50.315077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.315618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.316099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.421929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:50.424290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.426333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f094-8447-44ad-b237-7ac5bf817366 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.426333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f094-8447-44ad-b237-7ac5bf817366 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.426621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c70f094-8447-44ad-b237-7ac5bf817366 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.431002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 687/1367] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:50 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:50.436464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d443-3ae7-4437-b8e5-920f6c0d7f28 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.437010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d443-3ae7-4437-b8e5-920f6c0d7f28 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.437486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a43d443-3ae7-4437-b8e5-920f6c0d7f28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.478864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:50.536689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0a43d443-3ae7-4437-b8e5-920f6c0d7f28 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:50.538242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0a43d443-3ae7-4437-b8e5-920f6c0d7f28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.634512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 681/1368] 10.222.0.97 () {58 vars in 1059 bytes} [Thu Sep 8 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:50.644297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:50.661412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b4a6e2-72ab-4a41-a622-495a3dd62ca1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:50.712793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 688/1369] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:50.820631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:50.821119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:50.821617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:50.821817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:50.824733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:50.849955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:50.997807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:51.001700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6ae005-3f75-4bef-bc95-2c78ac041986 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:51.044996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 682/1370] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:51.046773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.047240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.047716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.050491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 689/1371] 10.222.0.97 () {54 vars in 964 bytes} [Thu Sep 8 13:53:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:53:51.062204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c9da-76fc-4315-9c38-e83202d8d6ca None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.062698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c9da-76fc-4315-9c38-e83202d8d6ca None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.063155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e6c9da-76fc-4315-9c38-e83202d8d6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.177348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c4e6c9da-76fc-4315-9c38-e83202d8d6ca None None] MFA Rules not processed for user `eb8f5ef6eb4948edbf947eacffea57bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:51.178723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c4e6c9da-76fc-4315-9c38-e83202d8d6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.296279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 683/1372] 10.222.0.97 () {58 vars in 1061 bytes} [Thu Sep 8 13:53:51 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:51.306621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:51.308610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.339561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:51.452673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8cdfe-9f59-48a8-b675-5513b03e2fd2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:51.588478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:51.591523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:51.592332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:51.592934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:51.596758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:51.723582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:51.815367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:51.818386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bfa91d-084c-4e62-9d5b-e470de04dc65 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:51.858996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 690/1373] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 553 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:53:53.868475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.869234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.869234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.871058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 684/1374] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:53.877364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf62-d22b-4a4d-95f4-0748ca4a9ef9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:53.877983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf62-d22b-4a4d-95f4-0748ca4a9ef9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:53.877983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ebf62-d22b-4a4d-95f4-0748ca4a9ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:53.959429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4c7ebf62-d22b-4a4d-95f4-0748ca4a9ef9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:53.960844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4c7ebf62-d22b-4a4d-95f4-0748ca4a9ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.156746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 691/1375] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:53:54.500739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:54.502942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:54.515830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:54.724529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf98614-ec5d-4c95-b990-0a51151acd71 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:54.864764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:54.914082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:54.915543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:54.915543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:54.915543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:54.917882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:55.062938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:55.062938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691856a8-cc5a-4f55-a74c-62e78c683cf1 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:55.090519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 685/1376] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:53:58.125492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.126057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.126057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.127424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 692/1377] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:53:58.133548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9a23-925e-4956-8a29-8211b8b0d28f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:58.134120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9a23-925e-4956-8a29-8211b8b0d28f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:58.134219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8a9a23-925e-4956-8a29-8211b8b0d28f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:58.228040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-cb8a9a23-925e-4956-8a29-8211b8b0d28f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:53:58.229456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cb8a9a23-925e-4956-8a29-8211b8b0d28f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.294582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:58.366811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 686/1378] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:53:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:53:58.721748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:53:58.723937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:58.787277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:58.988172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2911480c-3b86-459d-af7c-087237b2b744 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:53:59.190659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:53:59.207252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:53:59.209414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:53:59.209924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:53:59.210409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:53:59.214033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:53:59.351068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:53:59.352572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f582ef-00a9-4826-b238-7ea68d639a63 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:53:59.382661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 693/1379] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:53:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 657 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:02.724729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.725302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.725600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.727099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 687/1380] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:54:02.733570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a5160-be77-4461-81f2-f36bc9e409fd None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:02.734586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a5160-be77-4461-81f2-f36bc9e409fd None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:02.734872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a5160-be77-4461-81f2-f36bc9e409fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:02.855083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-552a5160-be77-4461-81f2-f36bc9e409fd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:02.858187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-552a5160-be77-4461-81f2-f36bc9e409fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:02.953920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.000932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 694/1381] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:03.305508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:03.308793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.549933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd05487-cce3-4589-ad89-d89aebea5ce8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:03.594548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:03.708562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:03.709211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:03.709505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:03.709776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:03.711566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:03.863644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:03.865268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8ec705-5614-4a08-b7f0-1e6739ed87e3 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:03.887059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 688/1382] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:07.238791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.239422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.239729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.242474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 695/1383] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:07 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:07.251812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681a57-d2c9-499b-9cc0-849693bfb322 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:07.252742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681a57-d2c9-499b-9cc0-849693bfb322 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:07.252742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8681a57-d2c9-499b-9cc0-849693bfb322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:07.305335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:07.340648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c8681a57-d2c9-499b-9cc0-849693bfb322 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:07.342241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c8681a57-d2c9-499b-9cc0-849693bfb322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:07.489435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 689/1384] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:07.816785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:07.818476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:08.052338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083cfef6-7575-4054-8ae2-5d1d43b4f96f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:08.141301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:08.241990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:08.243067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:08.243067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:08.243391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:08.245421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:08.372604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:08.374365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136368f-757f-4048-b0ba-850862b06c60 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:08.400948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 696/1385] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:09.285478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414c70-3fc7-4b22-8538-c653df26e269 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.286145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414c70-3fc7-4b22-8538-c653df26e269 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.286145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a414c70-3fc7-4b22-8538-c653df26e269 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.287592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 690/1386] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:09.293954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0ae18-456e-49ad-9e81-f68554cf63d3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.294574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0ae18-456e-49ad-9e81-f68554cf63d3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.294574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c0ae18-456e-49ad-9e81-f68554cf63d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.389453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-57c0ae18-456e-49ad-9e81-f68554cf63d3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:09.391538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-57c0ae18-456e-49ad-9e81-f68554cf63d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:09.395921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:09.588290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 697/1387] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:09.908754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.909253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.909670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:09.911367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 691/1388] 10.222.0.97 () {54 vars in 961 bytes} [Thu Sep 8 13:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:54:09.918071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfa7e-45f2-49a1-bca7-9f3d378ee3d3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:09.918459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfa7e-45f2-49a1-bca7-9f3d378ee3d3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:09.918827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4bfa7e-45f2-49a1-bca7-9f3d378ee3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.042022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-bc4bfa7e-45f2-49a1-bca7-9f3d378ee3d3 None None] MFA Rules not processed for user `9f7790a1b25740fba9cb92c1fa53ba2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:10.044023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bc4bfa7e-45f2-49a1-bca7-9f3d378ee3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:10.147127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:10.150705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 698/1389] 10.222.0.97 () {58 vars in 1058 bytes} [Thu Sep 8 13:54:09 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:10.160803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:10.162425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:10.292429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad3c7d-594d-46b0-a01a-0db42c2683c8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:10.306801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:10.392171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb17119-2740-4088-b531-410350267701 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:10.393238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17119-2740-4088-b531-410350267701 service glance] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:10.393783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17119-2740-4088-b531-410350267701 service glance] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:10.394266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb17119-2740-4088-b531-410350267701 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:10.398124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8eb17119-2740-4088-b531-410350267701 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:10.566201 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb17119-2740-4088-b531-410350267701 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:10.568415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb17119-2740-4088-b531-410350267701 service glance] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:10.604526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 692/1390] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:11.158549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.159254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.159254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:11.160644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 699/1391] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:11.167125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa099c3e-47a5-4579-b043-5d599068a6bf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.167532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa099c3e-47a5-4579-b043-5d599068a6bf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.167686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa099c3e-47a5-4579-b043-5d599068a6bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:11.240332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:11.265664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-aa099c3e-47a5-4579-b043-5d599068a6bf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:11.266946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-aa099c3e-47a5-4579-b043-5d599068a6bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:11.451207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 693/1392] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:11.755813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:11.757472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:11.932098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbefd63-2017-493c-ad98-b3c233e012a2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:11.993290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634f19-271a-4c85-97a5-4c07f54bc565 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:11.993736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634f19-271a-4c85-97a5-4c07f54bc565 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:11.994163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d634f19-271a-4c85-97a5-4c07f54bc565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.060872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:12.086510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4d634f19-271a-4c85-97a5-4c07f54bc565 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:12.087806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4d634f19-271a-4c85-97a5-4c07f54bc565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.130285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.130911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.131189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.131463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.133247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.190703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:12.222039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 694/1393] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 13:54:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:12.262490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.264490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.290993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73481cab-d971-4ded-b379-7ca6cf690b32 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.291576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73481cab-d971-4ded-b379-7ca6cf690b32 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.292577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73481cab-d971-4ded-b379-7ca6cf690b32 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.294542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 695/1394] 10.222.0.97 () {54 vars in 943 bytes} [Thu Sep 8 13:54:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:54:12.299567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 700/1395] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:12.303181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902b3c-338f-447a-b51e-59c675088891 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.303580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902b3c-338f-447a-b51e-59c675088891 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.304078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da902b3c-338f-447a-b51e-59c675088891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.400515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-da902b3c-338f-447a-b51e-59c675088891 None None] MFA Rules not processed for user `71620f9e7c3147829e384a5cfae55f07`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:12.402529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-da902b3c-338f-447a-b51e-59c675088891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.546387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 696/1396] 10.222.0.97 () {58 vars in 1040 bytes} [Thu Sep 8 13:54:12 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:12.556992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:12.558424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.670233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:12.674293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d874386f-3258-4c6b-ab7f-60a08cabe07c service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:12.820340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:12.821163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:12.821561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:12.822191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:12.825017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:12.953706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:12.955372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0af2429-7be9-44e8-9347-f8fadd685e42 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:12.965540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 701/1397] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 13:54:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 13:54:15.362016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.362727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.362727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.363936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 697/1398] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:15.370803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c2a99-ad2f-43c0-a078-ac58c280a8e4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:15.371519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c2a99-ad2f-43c0-a078-ac58c280a8e4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:15.371732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023c2a99-ad2f-43c0-a078-ac58c280a8e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:15.428739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:15.470038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-023c2a99-ad2f-43c0-a078-ac58c280a8e4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:15.471692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-023c2a99-ad2f-43c0-a078-ac58c280a8e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.622435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 702/1399] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:15.933427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:15.941715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:15.996156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:16.209673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ff8e9e-ca45-4a8b-af15-23f9a4252625 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:16.475076 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:16.476110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:16.476320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:16.476469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:16.478534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:16.484030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:16.660458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:16.663348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56425c7b-6be1-47e5-882b-ba0b5d5c271b service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:16.707786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 698/1400] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 776 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:19.490967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:19.491586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:19.491586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:19.493158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 703/1401] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:19.499707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b404-b981-4d6f-a2c3-f523c08d8027 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:19.500458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b404-b981-4d6f-a2c3-f523c08d8027 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:19.500458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b3b404-b981-4d6f-a2c3-f523c08d8027 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:19.586524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-46b3b404-b981-4d6f-a2c3-f523c08d8027 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:19.587805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-46b3b404-b981-4d6f-a2c3-f523c08d8027 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:19.733028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:19.773081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 699/1402] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:20.338749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:20.342064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:20.582771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:20.643624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3679be-fef8-4f0c-9aa6-afd7e4337987 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:20.843449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:20.843969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:20.844312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:20.844810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:20.846537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:20.959281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:20.985170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:20.987136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146c4ece-0189-439a-90c8-1c9e33520975 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:21.021008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 704/1403] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:24.243150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:24.243753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:24.244057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:24.252234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 700/1404] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:24.255284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ccb3b-41e9-4a07-a6ff-897b83b90914 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:24.255871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ccb3b-41e9-4a07-a6ff-897b83b90914 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:24.256306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ccb3b-41e9-4a07-a6ff-897b83b90914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:24.419793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b26ccb3b-41e9-4a07-a6ff-897b83b90914 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:24.421931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b26ccb3b-41e9-4a07-a6ff-897b83b90914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:24.628198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 705/1405] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:25.101379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:25.105008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.459092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996369d-c276-49ce-822a-5b4185fb1e8c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:25.570013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:25.657967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:25.658904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:25.659530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:25.659821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:25.661761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:25.800463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:25.802211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bc1cd9-b0e2-47e8-8619-941820ad9841 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:25.832691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 701/1406] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 732 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:28.787217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.787939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.787939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.789122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 706/1407] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:54:28.795299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982a58-fbad-4e9a-862b-7791ba95eb78 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:28.795299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982a58-fbad-4e9a-862b-7791ba95eb78 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:28.796090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db982a58-fbad-4e9a-862b-7791ba95eb78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:28.871095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-db982a58-fbad-4e9a-862b-7791ba95eb78 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:28.872351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-db982a58-fbad-4e9a-862b-7791ba95eb78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:28.892121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.019612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 702/1408] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:29.317705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:29.320175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.387853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.523849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b949d45f-3172-49c6-8593-7adbfb3806e9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:29.700690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:29.701971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:29.702472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:29.702962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:29.705325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:29.725212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:29.880241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:29.884578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf2d602-e492-4a96-a693-a71f4137c38f service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:29.917201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 707/1409] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:39.039173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=35 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:39.914480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=40 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:40.933500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540286-294c-4314-879b-cddb72416615 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:40.934097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540286-294c-4314-879b-cddb72416615 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:40.934197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40540286-294c-4314-879b-cddb72416615 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:40.935629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 703/1410] 10.222.0.97 () {54 vars in 918 bytes} [Thu Sep 8 13:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:40.945021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562998c-307b-4d62-81fc-78694f3629ac None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:40.945021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562998c-307b-4d62-81fc-78694f3629ac None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:40.945021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1562998c-307b-4d62-81fc-78694f3629ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:41.031712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1562998c-307b-4d62-81fc-78694f3629ac None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:41.033432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1562998c-307b-4d62-81fc-78694f3629ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:41.161059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 708/1411] 10.222.0.97 () {58 vars in 1015 bytes} [Thu Sep 8 13:54:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:41.178367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7a8d-2903-427a-a9ee-7ebbf149d15d None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:41.178650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7a8d-2903-427a-a9ee-7ebbf149d15d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:41.178926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7a8d-2903-427a-a9ee-7ebbf149d15d None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:41.180335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 704/1412] 10.222.0.97 () {54 vars in 918 bytes} [Thu Sep 8 13:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:54:41.187195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81814-c9c0-46c5-ae9b-6d441056a035 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:41.187712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81814-c9c0-46c5-ae9b-6d441056a035 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:41.188239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f81814-c9c0-46c5-ae9b-6d441056a035 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:41.277030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:41.280530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-01f81814-c9c0-46c5-ae9b-6d441056a035 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:41.281781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-01f81814-c9c0-46c5-ae9b-6d441056a035 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:41.414376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 709/1413] 10.222.0.97 () {58 vars in 1015 bytes} [Thu Sep 8 13:54:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:44.196893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:44.197503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:44.197503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:44.199493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:54:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:44.205868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50185e-d239-48ff-9fcf-4cbd4305bf1f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:44.206453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50185e-d239-48ff-9fcf-4cbd4305bf1f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:44.206453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50185e-d239-48ff-9fcf-4cbd4305bf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:44.289530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3b50185e-d239-48ff-9fcf-4cbd4305bf1f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:44.290744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3b50185e-d239-48ff-9fcf-4cbd4305bf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:44.357997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:44.410797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:44.717744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:44.719474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:44.922722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055ca0cc-1aa3-40d9-82c1-a60658b75a32 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:45.023128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:45.068295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:45.068901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:45.069172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:45.069440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:45.071063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:45.179298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:45.180940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06785e4b-082d-4326-9781-ba2f9919c52e service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:45.201867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 706/1416] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:46.431208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:46.431997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:46.431997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:46.433188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:46.440570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ae1d7-379f-4506-9fbc-63f12760696a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:46.441462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ae1d7-379f-4506-9fbc-63f12760696a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:46.441462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455ae1d7-379f-4506-9fbc-63f12760696a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:46.545113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-455ae1d7-379f-4506-9fbc-63f12760696a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:46.546361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-455ae1d7-379f-4506-9fbc-63f12760696a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:46.577038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:46.654671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:46.825003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:46.830374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:46.993073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708ec6e-fe88-486e-a5a2-4ac3fba1ae74 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:47.040580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:47.126425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:47.126921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:47.127192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:47.127457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:47.129195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:47.240500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:47.242127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8abbd1d-2ac1-4b72-8dcc-86d163a52801 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:47.264622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 712/1419] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:48.472065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:48.472670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:48.472670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:48.474358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:48.482417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25160-92f4-4dfa-9ea3-be23d402aa28 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:48.483029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25160-92f4-4dfa-9ea3-be23d402aa28 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:48.483029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c25160-92f4-4dfa-9ea3-be23d402aa28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:48.551455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:48.578615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-75c25160-92f4-4dfa-9ea3-be23d402aa28 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:48.579736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-75c25160-92f4-4dfa-9ea3-be23d402aa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:48.686776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:48.844263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:48.845946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.008633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd21246f-3a03-4a3a-ada1-c5bab946dc73 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:49.023464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:49.135049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:49.136136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:49.136440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:49.136707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:49.138305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:49.233159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:49.234815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95bc5d10-164d-47c6-ab6a-97adc6cfdc67 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:49.255211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 709/1422] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:50.416601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.417465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.417625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.419054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:50.427052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c18cd-69e1-40d6-aa00-4840c14c7f4d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:50.427627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c18cd-69e1-40d6-aa00-4840c14c7f4d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:50.427627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c18cd-69e1-40d6-aa00-4840c14c7f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:50.453030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:50.520479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c93c18cd-69e1-40d6-aa00-4840c14c7f4d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:50.521719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c93c18cd-69e1-40d6-aa00-4840c14c7f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:50.632596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:50.790285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:50.791890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:50.919985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:50.965591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773e81fa-7afa-4038-88b2-573fe0fdd588 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:51.104637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:51.105174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:51.105441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:51.105715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:51.107455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:51.194965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:51.212744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:51.214585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c4eed-a25f-4208-b100-adb43ff14bcf service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:51.250700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 715/1425] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:52.548801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:52.549416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:52.549416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:52.550875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:52.558889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40789bd-b269-4917-bf8d-22ac79294a93 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:52.559610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40789bd-b269-4917-bf8d-22ac79294a93 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:52.559610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40789bd-b269-4917-bf8d-22ac79294a93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:52.646294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d40789bd-b269-4917-bf8d-22ac79294a93 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:52.647433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d40789bd-b269-4917-bf8d-22ac79294a93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:52.760832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:52.919324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:52.921245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:53.047265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:53.129342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcd1a4-8b4a-41dc-b315-672e16710ce6 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:53.262739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:53.263351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:53.263639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:53.263891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:53.265575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:53.318486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:53.362630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:53.364289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88dda806-ffd2-4c85-80c7-7035ffbddfac service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:53.386144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 712/1428] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 468 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:54:54.608936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.609615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.609615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.610925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:54.618787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922009-0dc5-4d00-b8c1-0108b12177e4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:54.619421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922009-0dc5-4d00-b8c1-0108b12177e4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:54.619421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3922009-0dc5-4d00-b8c1-0108b12177e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:54.715590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f3922009-0dc5-4d00-b8c1-0108b12177e4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:54.716887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f3922009-0dc5-4d00-b8c1-0108b12177e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:54.830091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:54:54.990322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:54.991990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:55.053410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:55.202509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645208f-8c1d-440e-bd99-3e349bf7d9cc None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:55.346539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:55.347603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:55.348108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:55.348655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:55.351748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:55.365906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:55.518155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:55.519770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00439fe-e79e-4c6c-a9d8-fced9e72073e service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:55.539142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 718/1431] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:56.752455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.753043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.753043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.754463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:56.762634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c76c68-a8e5-48ed-aa2e-4788b86d1152 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:56.763239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c76c68-a8e5-48ed-aa2e-4788b86d1152 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:56.763239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c76c68-a8e5-48ed-aa2e-4788b86d1152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:56.846589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d9c76c68-a8e5-48ed-aa2e-4788b86d1152 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:56.847809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d9c76c68-a8e5-48ed-aa2e-4788b86d1152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:56.924269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:56.957608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:57.116679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:57.118259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:57.139779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:57.298405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2461ee-d270-4227-8b33-eb13acbd66fd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:57.403379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:57.431668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:57.432386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:57.432662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:57.432933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:57.434623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:57.530593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:57.532391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd26e24-f21d-455a-99de-8d0f54f5e68f service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:57.556753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 715/1434] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:54:58.851497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:58.852103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:58.852103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:58.853483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:54:58.861055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f502-d635-4ca1-91f8-0037c29bde61 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:58.861606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f502-d635-4ca1-91f8-0037c29bde61 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:58.861703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5629f502-d635-4ca1-91f8-0037c29bde61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:58.951995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5629f502-d635-4ca1-91f8-0037c29bde61 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:54:58.951995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5629f502-d635-4ca1-91f8-0037c29bde61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:58.998566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:59.058867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:54:59.217902 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:54:59.219378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:59.413913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44861c2c-8221-4f39-9f0b-23a7dbbb0a0a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:54:59.477522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:54:59.540781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:54:59.541406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:54:59.541717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:54:59.541988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:54:59.543673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:54:59.652019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:54:59.653691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a69a9-0a57-468e-aaa1-49ccc52bf97c service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:54:59.678624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 721/1437] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:54:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:01.067873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.068627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.068627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.069837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:01.076975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c678-bce6-4ebd-930f-1f2d2301726c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.077527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c678-bce6-4ebd-930f-1f2d2301726c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.077527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f7c678-bce6-4ebd-930f-1f2d2301726c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.184026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-84f7c678-bce6-4ebd-930f-1f2d2301726c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:01.185288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-84f7c678-bce6-4ebd-930f-1f2d2301726c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:01.187510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:01.306580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:01.466618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:01.468347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:01.655579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415fc29d-572c-4b17-9c66-b7e15ddc9d3a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:01.699043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:01.806764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:01.807415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:01.807696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:01.807972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:01.809661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:01.910433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:01.911939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb1498c-c27b-407e-84d8-b381cdb86e19 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:01.931056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 718/1440] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:04.136747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:04.137342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:04.137342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:04.138637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:04.144725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fadf33-75fb-4b16-a121-3afa67e91da2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:04.145266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fadf33-75fb-4b16-a121-3afa67e91da2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:04.145266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fadf33-75fb-4b16-a121-3afa67e91da2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:04.189850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:04.219391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-05fadf33-75fb-4b16-a121-3afa67e91da2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:04.220656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-05fadf33-75fb-4b16-a121-3afa67e91da2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:04.337805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:04.632858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:04.634742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:04.774345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fa58b-3ed5-4dba-b83b-75920a04caed None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:04.812562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:04.863762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:04.864393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:04.864681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:04.864948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:04.866583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:04.987983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:04.993055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f89ad7-93db-4a09-aead-c64f89efc32d service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:05.009453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 724/1443] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:07.419124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:07.419778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:07.420021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:07.421911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:07.428959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42602e-1ab1-4e62-94d5-0b16cc891e2b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:07.429507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42602e-1ab1-4e62-94d5-0b16cc891e2b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:07.429846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f42602e-1ab1-4e62-94d5-0b16cc891e2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:07.503229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4f42602e-1ab1-4e62-94d5-0b16cc891e2b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:07.504515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4f42602e-1ab1-4e62-94d5-0b16cc891e2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:07.520112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:07.620926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:07.911927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:07.913651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:08.044608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9747c7-cdad-412e-b9c6-66eb9bda4228 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:08.060455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:08.138864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:08.139476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:08.139758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:08.140034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:08.141982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:08.305406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:08.306886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0969414f-fdd7-4932-b04e-fc8ba6a82d4c service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:08.329217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 721/1446] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:11.128036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.128706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.128706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.130106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:11.136870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e69-862c-4b74-959d-adb34e151fd0 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.137412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e69-862c-4b74-959d-adb34e151fd0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.137560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f95e69-862c-4b74-959d-adb34e151fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.196902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:11.223927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-59f95e69-862c-4b74-959d-adb34e151fd0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:11.229844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-59f95e69-862c-4b74-959d-adb34e151fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.336370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:11.644025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:11.645932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.772078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a3d25-9d9b-4e52-951a-ddfd97e5eb3b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:11.822174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:11.864546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:11.865166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:11.865441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:11.865709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:11.867407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:11.995815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:11.998325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a96d3ac-4f6d-4758-bc54-db6e6b9b5dd7 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:12.034832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 727/1449] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:14.597805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:14.598406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:14.598406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:14.599844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:55:14.607160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d3e8-d5b0-470c-9be1-112c07961bb7 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:14.607974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d3e8-d5b0-470c-9be1-112c07961bb7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:14.607974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce00d3e8-d5b0-470c-9be1-112c07961bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:14.703305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ce00d3e8-d5b0-470c-9be1-112c07961bb7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:14.705463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ce00d3e8-d5b0-470c-9be1-112c07961bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:14.749698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:14.860950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:15.162113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:15.163691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:15.288235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884cd2ef-c1d8-4e72-97d2-8a5557e0bb9e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:15.311459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:15.370650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:15.371301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:15.371573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:15.371841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:15.373576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:15.501725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:15.503192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b28559-c791-4c05-ab64-9d38b4e312ad service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:15.525301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 724/1452] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:17.910055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:17.910659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:17.910659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:17.912639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:17.919432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f948-4730-44e0-b734-ff62d90449ad None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:17.920042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f948-4730-44e0-b734-ff62d90449ad None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:17.920042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a2f948-4730-44e0-b734-ff62d90449ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:18.003534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:18.008169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-78a2f948-4730-44e0-b734-ff62d90449ad None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:18.009438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-78a2f948-4730-44e0-b734-ff62d90449ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:18.116305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:18.401851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:18.403431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:18.525056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd3124-e0e8-4307-bf59-f353250d1760 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:18.580781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:18.608567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:18.609260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:18.609548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:18.609886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:18.611619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:18.750054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:18.751544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22971c0f-cfac-49dd-9108-27525b212ab6 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:18.773746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 730/1455] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:21.127023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:21.127643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:21.127643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:21.129291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:55:21.135526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ff82-2fd6-4659-88a9-090faf0df95c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:21.136174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ff82-2fd6-4659-88a9-090faf0df95c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:21.136330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f8ff82-2fd6-4659-88a9-090faf0df95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:21.219298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-34f8ff82-2fd6-4659-88a9-090faf0df95c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:21.220582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-34f8ff82-2fd6-4659-88a9-090faf0df95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:21.256461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:21.336131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:21.641439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:21.643147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:21.812121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf71072-f11b-4445-9c31-0564b7559bb0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:21.852569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:21.917641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:21.918005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:21.918167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:21.918319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:21.920459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:22.088506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:22.090306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51817b2c-0fd5-465f-9045-77fd6ca375d6 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:22.116743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 727/1458] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:24.513352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.513935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.513935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.515454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:55:24.521854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6d6b-1baa-4fd4-ac5d-de6d90e6a640 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:24.522413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6d6b-1baa-4fd4-ac5d-de6d90e6a640 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:24.522508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dd6d6b-1baa-4fd4-ac5d-de6d90e6a640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:24.605001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b1dd6d6b-1baa-4fd4-ac5d-de6d90e6a640 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:24.606245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b1dd6d6b-1baa-4fd4-ac5d-de6d90e6a640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:24.616337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:24.768124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:25.065290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:25.067000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:25.211582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7487a7d-6cdc-4151-8988-50da98544616 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:25.281793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:25.305532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:25.306216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:25.306437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:25.306710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:25.309270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:25.477524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:25.481375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ab3e51-4856-425d-8b5c-867ee1bc7273 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:25.518921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 733/1461] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:55:27.932813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.933564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.933564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:27.935109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:27.942491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00534b67-2714-423c-ba6d-3c30f76d338e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:27.945981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00534b67-2714-423c-ba6d-3c30f76d338e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:27.945981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00534b67-2714-423c-ba6d-3c30f76d338e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.073679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-00534b67-2714-423c-ba6d-3c30f76d338e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:28.075719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-00534b67-2714-423c-ba6d-3c30f76d338e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.130622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:28.213096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:28.505063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:28.506910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.637425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75775bba-64b1-43ca-bad8-ab5828037929 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:28.687050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:28.732364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:28.735403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:28.735403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:28.735403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:28.737061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:28.880361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:28.881846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1c7ffd-7665-4f97-9366-e0f503e803b0 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:28.904003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 730/1464] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:31.371623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.372417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.372417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.373781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:31.380002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975c1e-c321-4ff6-b505-47287335c6cb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:31.380830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975c1e-c321-4ff6-b505-47287335c6cb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:31.380830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27975c1e-c321-4ff6-b505-47287335c6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:31.461591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-27975c1e-c321-4ff6-b505-47287335c6cb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:31.463011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-27975c1e-c321-4ff6-b505-47287335c6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:31.486654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:31.581292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:31.869365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:31.871482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:31.996295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd4e4e0-88e4-40b4-b056-b134141ecfca None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:32.079622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:32.087118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:32.087811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:32.088094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:32.088421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:32.090107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:32.216294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:32.217897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c1ed4a-a48d-4a5b-8bb1-4d9fdbd68157 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:32.240172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 736/1467] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:34.559452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.560077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.560077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.561419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:34.567360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56454fc-c059-4788-8f73-7128e6c2e14e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:34.567824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56454fc-c059-4788-8f73-7128e6c2e14e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:34.567932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56454fc-c059-4788-8f73-7128e6c2e14e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:34.645385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d56454fc-c059-4788-8f73-7128e6c2e14e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:34.646563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d56454fc-c059-4788-8f73-7128e6c2e14e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:34.703768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:34.763776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:55:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:35.061269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:35.062816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:35.172355 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58d175-7b97-4d7d-b5d0-23c5c652b7e0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:35.224362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:35.252752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:35.253377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:35.253655 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:35.253929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:35.255650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:35.382981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:35.394942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecc86ae-73ec-4fb6-a9be-35c2df69b439 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:35.409624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 733/1470] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:55:44.767504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=22 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:45.407028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=66 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:56.642741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859041-82f2-470a-b84b-68ef86ea4411 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:56.643307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859041-82f2-470a-b84b-68ef86ea4411 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:56.643307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9859041-82f2-470a-b84b-68ef86ea4411 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:56.644626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 738/1471] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:55:56.652010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a14ae-192c-4e3d-9f41-44cbb6a5e633 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:56.652651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a14ae-192c-4e3d-9f41-44cbb6a5e633 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:56.652997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44a14ae-192c-4e3d-9f41-44cbb6a5e633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:56.753685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a44a14ae-192c-4e3d-9f41-44cbb6a5e633 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:56.755187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a44a14ae-192c-4e3d-9f41-44cbb6a5e633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:56.872441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 734/1472] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:55:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:55:57.181694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.181979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.182860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.184250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 739/1473] 10.222.0.97 () {54 vars in 961 bytes} [Thu Sep 8 13:55:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:55:57.190090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e620d3-5611-49c6-b8f2-e857298bcf99 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.190376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e620d3-5611-49c6-b8f2-e857298bcf99 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.190649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e620d3-5611-49c6-b8f2-e857298bcf99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.288465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f7e620d3-5611-49c6-b8f2-e857298bcf99 None None] MFA Rules not processed for user `9f7790a1b25740fba9cb92c1fa53ba2e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:55:57.289791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f7e620d3-5611-49c6-b8f2-e857298bcf99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.316909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:57.385065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 735/1474] 10.222.0.97 () {58 vars in 1058 bytes} [Thu Sep 8 13:55:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:55:57.393448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:55:57.395143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.524765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f37d3-c1d5-43df-a8dd-c9e711cd9b39 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:55:57.615217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:55:57.615889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:55:57.616299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:55:57.616665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:55:57.618660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:55:57.687274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:55:57.762843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:55:57.764635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527afb21-b7fc-4738-ab13-8fd4ba4ba984 service glance] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:55:57.788039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 740/1475] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:55:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:56:00.330073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.330689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.330803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.332530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 736/1476] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:00.343492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265443a-35e8-475d-b5e4-0c750aa77c44 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:00.344077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265443a-35e8-475d-b5e4-0c750aa77c44 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:00.344701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3265443a-35e8-475d-b5e4-0c750aa77c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:00.438049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3265443a-35e8-475d-b5e4-0c750aa77c44 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:00.439473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3265443a-35e8-475d-b5e4-0c750aa77c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:00.559063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 741/1477] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:00.837403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:00.839113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:01.065553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad214efd-1b1b-47fd-8fc1-d75571113bc2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:01.077785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:01.136633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:01.137251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:01.137528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:01.137798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:01.138478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:01.140328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:56:01.140328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 13:56:01.140328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 13:56:01.141656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01d998-5c5d-4ce1-9d40-8721bdf0a5f8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:01.168946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 737/1478] 10.222.0.97 () {56 vars in 1108 bytes} [Thu Sep 8 13:56:00 2022] GET /identity/v3/projects => generated 2553 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 13:56:03.586269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.586269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.586269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.587512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 742/1479] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:03.593615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c02-057a-4789-90e5-ca6418c97761 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:03.594171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c02-057a-4789-90e5-ca6418c97761 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:03.594275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fc7c02-057a-4789-90e5-ca6418c97761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:03.671520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b4fc7c02-057a-4789-90e5-ca6418c97761 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:03.672785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b4fc7c02-057a-4789-90e5-ca6418c97761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:03.789277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 738/1480] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:04.089689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:04.091385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.101872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:04.268021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd64299-17b7-48d0-9fac-ba63977b9ade None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:04.497750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:04.538215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:04.538824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:04.539106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:04.539363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:04.541106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:04.649622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:04.652032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3997c-f881-4b75-ac9e-7c6d94ab683a service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:04.675324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 743/1481] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:07.562410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a0c-bb37-4031-a614-840143494372 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:07.563243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a0c-bb37-4031-a614-840143494372 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:07.563441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10799a0c-bb37-4031-a614-840143494372 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:07.565061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 739/1482] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:07.575311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820e6e-f05c-48e1-9bcc-98a36f1f8b3b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:07.575871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820e6e-f05c-48e1-9bcc-98a36f1f8b3b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:07.576238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2820e6e-f05c-48e1-9bcc-98a36f1f8b3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:07.674244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d2820e6e-f05c-48e1-9bcc-98a36f1f8b3b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:07.675738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d2820e6e-f05c-48e1-9bcc-98a36f1f8b3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:07.733327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:07.810471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 744/1483] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:56:08.107257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799a0c-bb37-4031-a614-840143494372 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:08.110172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-10799a0c-bb37-4031-a614-840143494372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.151135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:08.250162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10799a0c-bb37-4031-a614-840143494372 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:08.360296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:08.361259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:08.361755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:08.362258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:08.365610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:08.472405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:08.515516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:08.517019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8550301d-f520-4c10-9cbf-d128b81267e3 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:08.539544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 740/1484] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:10.891516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:10.892111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:10.892111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:10.893480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 745/1485] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:10.899772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995323-a469-42c8-b640-8b6de708e96f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:10.900668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995323-a469-42c8-b640-8b6de708e96f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:10.900668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13995323-a469-42c8-b640-8b6de708e96f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:10.984897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-13995323-a469-42c8-b640-8b6de708e96f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:10.986161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-13995323-a469-42c8-b640-8b6de708e96f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:11.096731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 741/1486] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:11.388738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:11.390360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:11.540599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be524531-64d8-462c-acae-e92bdf93a89c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:11.630233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:11.631768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:11.632630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:11.633038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:11.633434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:11.635327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:11.771957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:11.773467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebf0a85-1f5b-4bc0-bb9b-67fbd39e43c0 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:11.799311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 746/1487] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:14.340825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.341451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.341451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.342738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 742/1488] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:56:14.349013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a6f8-7520-4507-908b-4fcd63c687b3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:14.349889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a6f8-7520-4507-908b-4fcd63c687b3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:14.349889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab2a6f8-7520-4507-908b-4fcd63c687b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:14.443549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0ab2a6f8-7520-4507-908b-4fcd63c687b3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:14.444850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0ab2a6f8-7520-4507-908b-4fcd63c687b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:14.505713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:14.564423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 747/1489] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:14.867857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:14.870898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:14.915941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:14.999457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e10cbe1-c723-46f7-ba96-43ecad2cc7ed None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:15.092004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:15.092658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:15.092942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:15.093214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:15.094898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:15.188907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:15.222001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:15.223516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2cea9c-de5e-40da-8639-a32ee264b7a1 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:15.246215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 743/1490] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:18.022597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.023251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.023251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.024527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 748/1491] 10.222.0.97 () {54 vars in 927 bytes} [Thu Sep 8 13:56:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:56:18.031036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec587-6e12-47f1-8e77-9ceed96e837e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.031820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec587-6e12-47f1-8e77-9ceed96e837e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.031820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2ec587-6e12-47f1-8e77-9ceed96e837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.106822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3c2ec587-6e12-47f1-8e77-9ceed96e837e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:56:18.108040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3c2ec587-6e12-47f1-8e77-9ceed96e837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:18.227145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 744/1492] 10.222.0.97 () {58 vars in 1024 bytes} [Thu Sep 8 13:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:56:18.526806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:56:18.528487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:18.672523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e941fb5-7c8a-43da-ae7a-8f12e484ce20 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:56:18.764975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:56:18.765582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:56:18.765857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:56:18.766130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:56:18.767907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:56:18.777941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:18.895757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:56:18.897299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db5791-d5b6-40ae-a02e-e0639179713e service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:56:18.918540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 749/1493] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:56:28.225279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=82 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:56:28.915334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=50 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:53.538832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.539550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.539550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.653100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:53.654417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:53.775016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 745/1494] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 13:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:53.784442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.785308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.785487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.817319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:53.819109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:53.832344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 750/1495] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 13:57:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:53.837896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:53.849032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d8c350-ba09-44f0-8835-31759b1398aa None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:53.854108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:53.854812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.855093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.855363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.856012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 13:57:53.856346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:53.857412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:53.857412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 13:57:53.857412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 13:57:53.858312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:53.872274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 746/1496] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 13:57:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:57:53.881520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:53.896666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3b8767-3a85-476f-a868-3cdccb3d3c73 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:53.903595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:53.904455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:53.904878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:53.905288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:53.906281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:53.909130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:53.909130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 13:57:53.909130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 13:57:53.911191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.072646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 751/1497] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 13:57:53 2022] POST /identity/v3/projects => generated 377 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 13:57:54.077633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.098345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6882a73-f8a7-47e0-b7bb-9f6df030751f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.150071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:54.165700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.166362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.166669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.166944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.168324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.169407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:54.169407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 13:57:54.169407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 13:57:54.170921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.403501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 747/1498] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 13:57:54 2022] POST /identity/v3/users => generated 289 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 13:57:54.408163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.422282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e293f9b7-6ef9-4a5a-a4ae-9f4d29359a97 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.429072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.429736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.430023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.430301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.430973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.432056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:54.432056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 13:57:54.432056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 13:57:54.433040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.448233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 752/1499] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:57:54.455010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.469055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a7d3b8-b111-4ffe-adf9-89c3f0a4de01 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.474120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.474918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.475246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.475681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.476450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.477543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:54.477543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 13:57:54.477543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 13:57:54.478583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8528d1-acdb-4a53-8a1c-a4d55448f35d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.495707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 748/1500] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 13:57:54 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 13:57:54.500608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.512564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e085f6a-5e06-47de-95a7-c61fc349dd96 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.517673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.518334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.518615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.518886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] PATH_INFO: `/v3/projects/7a1543a955354aeba430a176fa9a8ca8/users/6b50b3347b5644e7b12d56228e2c3ee9/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.537020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1543a955354aeba430a176fa9a8ca8, user_id=6b50b3347b5644e7b12d56228e2c3ee9, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.538080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:54.538080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 13:57:54.538080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 13:57:54.540039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa9859-f44a-45c2-995a-320031392714 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.646489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 753/1501] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 13:57:54 2022] PUT /identity/v3/projects/7a1543a955354aeba430a176fa9a8ca8/users/6b50b3347b5644e7b12d56228e2c3ee9/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:57:54.669887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0880dd-1580-4c8b-b877-0e7dbddf0e1f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.670724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0880dd-1580-4c8b-b877-0e7dbddf0e1f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.670724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0880dd-1580-4c8b-b877-0e7dbddf0e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.702972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ba0880dd-1580-4c8b-b877-0e7dbddf0e1f None None] MFA Rules not processed for user `6b50b3347b5644e7b12d56228e2c3ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:54.704180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ba0880dd-1580-4c8b-b877-0e7dbddf0e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:54.779506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 749/1502] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 13:57:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:57:54.794332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cbd-6164-4f9c-923b-e9ed15667d7d None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.794809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cbd-6164-4f9c-923b-e9ed15667d7d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.794906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a271cbd-6164-4f9c-923b-e9ed15667d7d None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.796252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 754/1503] 10.222.0.97 () {54 vars in 964 bytes} [Thu Sep 8 13:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:57:54.801388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b430-da60-4ea1-992b-b347e32bec7a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.801388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b430-da60-4ea1-992b-b347e32bec7a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.801672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0091b430-da60-4ea1-992b-b347e32bec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.863239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0091b430-da60-4ea1-992b-b347e32bec7a None None] MFA Rules not processed for user `eb8f5ef6eb4948edbf947eacffea57bb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:54.864416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0091b430-da60-4ea1-992b-b347e32bec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:54.938942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 750/1504] 10.222.0.97 () {58 vars in 1061 bytes} [Thu Sep 8 13:57:54 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:54.949208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271cbd-6164-4f9c-923b-e9ed15667d7d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:54.959907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a271cbd-6164-4f9c-923b-e9ed15667d7d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:54.963310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:54.963895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:54.964772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:54.965039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:54.973632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:54.975042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:54.984308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 755/1505] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:55.031998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194dbef-ef4f-46e7-8602-7caf69723067 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.032364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194dbef-ef4f-46e7-8602-7caf69723067 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.032725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7194dbef-ef4f-46e7-8602-7caf69723067 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.034105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 751/1506] 10.222.0.97 () {52 vars in 830 bytes} [Thu Sep 8 13:57:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:57:55.038443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:55.112630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e0006f-9b3d-4dd8-9841-59bd9c0284de service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:55.117571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:55.118246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:55.118521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:55.118792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] PATH_INFO: `/v3/users/6b50b3347b5644e7b12d56228e2c3ee9` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:55.120003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b50b3347b5644e7b12d56228e2c3ee9)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:55.121582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:55.121582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 13:57:55.121582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 13:57:55.122783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:55.765508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b50b3347b5644e7b12d56228e2c3ee9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:55.766378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b50b3347b5644e7b12d56228e2c3ee9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:55.783408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b50b3347b5644e7b12d56228e2c3ee9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:55.784456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b50b3347b5644e7b12d56228e2c3ee9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:55.815374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b50b3347b5644e7b12d56228e2c3ee9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:55.820639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:56.121942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 756/1507] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 13:57:55 2022] DELETE /identity/v3/users/6b50b3347b5644e7b12d56228e2c3ee9 => generated 0 bytes in 1084 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 13:57:56.126345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:56.126910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:56.126910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:56.147223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:57:56.148539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:56.158152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:57:56.209822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 752/1508] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 13:57:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:57:56.225516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:56.225516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:56.316651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d5471d-cd8e-4d57-b5e9-324686a8f8dc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:56.321760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:56.322460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:56.322761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:56.323011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:56.334160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:56.335042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc85425-cd1b-4b3f-98de-c85785ef8b83 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:56.345933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 757/1509] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:57:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:57:57.115189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:57:57.116912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:57:57.132523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b960ddb-580d-4c06-a478-5f07b1701c4c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:57:57.139135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:57:57.139822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:57:57.140315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:57:57.140792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] PATH_INFO: `/v3/projects/7a1543a955354aeba430a176fa9a8ca8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:57:57.142143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1543a955354aeba430a176fa9a8ca8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:57:57.143567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 13:57:57.143567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 13:57:57.143567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 13:57:57.145563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:57:57.327540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1543a955354aeba430a176fa9a8ca8', 'request_id': 'req-b26f8683-b5c2-4e39-8772-fe9304b3892c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:57.344731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1543a955354aeba430a176fa9a8ca8', 'request_id': 'req-b26f8683-b5c2-4e39-8772-fe9304b3892c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:57.352740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1543a955354aeba430a176fa9a8ca8', 'request_id': 'req-b26f8683-b5c2-4e39-8772-fe9304b3892c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:57.466075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] The token cache is being invalidate because project 7a1543a955354aeba430a176fa9a8ca8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 13:57:57.466306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:57.467102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b26f8683-b5c2-4e39-8772-fe9304b3892c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 13:57:57.469031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 753/1510] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 13:57:57 2022] DELETE /identity/v3/projects/7a1543a955354aeba430a176fa9a8ca8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 13:58:06.340987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=1,keystone:SELECT=37 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:58:07.461236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=47,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:30.653333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:30.654221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:30.654221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:30.655786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:59:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:30.661910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d2bc2-a742-4bc8-b3e9-528e8b7f2d7a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:30.662572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d2bc2-a742-4bc8-b3e9-528e8b7f2d7a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:30.662572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517d2bc2-a742-4bc8-b3e9-528e8b7f2d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:30.683435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-517d2bc2-a742-4bc8-b3e9-528e8b7f2d7a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:30.685230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-517d2bc2-a742-4bc8-b3e9-528e8b7f2d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:30.752015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:59:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:30.915654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:30.917789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:31.059824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe090b8f-684e-4a88-a725-6bb9c4abc786 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:31.065980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:31.066662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:31.067001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:31.067323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:31.077524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:31.079123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6058710d-c1f3-4c11-8459-0211253acee0 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:31.089997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 759/1513] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:32.239431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef96-ef96-4655-bed0-3ef77ba78b21 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.240101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef96-ef96-4655-bed0-3ef77ba78b21 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.240101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d709ef96-ef96-4655-bed0-3ef77ba78b21 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.241438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:32.248832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e30d7-d5ac-4538-a4ef-dd09513aca0f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.249423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e30d7-d5ac-4538-a4ef-dd09513aca0f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.249423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4e30d7-d5ac-4538-a4ef-dd09513aca0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.273029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ed4e30d7-d5ac-4538-a4ef-dd09513aca0f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:32.274275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ed4e30d7-d5ac-4538-a4ef-dd09513aca0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:32.283561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:59:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:32.450192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709ef96-ef96-4655-bed0-3ef77ba78b21 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:32.466238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d709ef96-ef96-4655-bed0-3ef77ba78b21 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:32.472037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:32.472693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.472973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.473246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.482899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:32.486025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61741b87-ead5-497e-acd2-c8fd98b5f943 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:32.495081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 756/1516] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:32.504661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45c12-2578-4aa9-9a79-80ec8b69096f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.505060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45c12-2578-4aa9-9a79-80ec8b69096f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.505395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea45c12-2578-4aa9-9a79-80ec8b69096f None None] PATH_INFO: `/v3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.507170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 761/1517] 10.222.0.77 () {54 vars in 925 bytes} [Thu Sep 8 13:59:32 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 13:59:32.515220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84e2a-403d-4395-ad5c-130411c7984f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:32.515630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84e2a-403d-4395-ad5c-130411c7984f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:32.515914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c84e2a-403d-4395-ad5c-130411c7984f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:32.584866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-74c84e2a-403d-4395-ad5c-130411c7984f None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:32.586124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-74c84e2a-403d-4395-ad5c-130411c7984f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:32.674371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 757/1518] 10.222.0.77 () {58 vars in 1015 bytes} [Thu Sep 8 13:59:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:41.888998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714edd-18ed-461c-8a85-fd091cd55d93 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.889611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714edd-18ed-461c-8a85-fd091cd55d93 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.889611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21714edd-18ed-461c-8a85-fd091cd55d93 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.891037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 762/1519] 10.222.0.97 () {54 vars in 961 bytes} [Thu Sep 8 13:59:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:41.896455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef880-06e0-4025-8216-150e477eaebf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.900404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef880-06e0-4025-8216-150e477eaebf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.900404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7ef880-06e0-4025-8216-150e477eaebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.920485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2d7ef880-06e0-4025-8216-150e477eaebf None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:41.921805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2d7ef880-06e0-4025-8216-150e477eaebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:41.930977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 758/1520] 10.222.0.97 () {58 vars in 1058 bytes} [Thu Sep 8 13:59:41 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:41.941427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21714edd-18ed-461c-8a85-fd091cd55d93 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:41.956777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21714edd-18ed-461c-8a85-fd091cd55d93 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:41.962338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:41.962962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:41.963247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:41.963522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:41.973889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:41.976457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0b255e-68d1-4cab-a596-bc077fcb1c6c service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:41.995082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 763/1521] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 13:59:44.065992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe725-5dd1-4ad6-b866-7433e368d363 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.066610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe725-5dd1-4ad6-b866-7433e368d363 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.066610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebe725-5dd1-4ad6-b866-7433e368d363 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.067895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:44.076104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5208-3bb1-4254-adbf-2a331322859b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.076672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5208-3bb1-4254-adbf-2a331322859b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.076672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e5208-3bb1-4254-adbf-2a331322859b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.097183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-860e5208-3bb1-4254-adbf-2a331322859b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:44.098481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-860e5208-3bb1-4254-adbf-2a331322859b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:44.107208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:59:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:44.263815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebe725-5dd1-4ad6-b866-7433e368d363 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:44.283280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebe725-5dd1-4ad6-b866-7433e368d363 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:44.290740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:44.291362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:44.291642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:44.291916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:44.302497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:44.304126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc70bcd8-0bbc-49ad-b599-6aac3d0a1b07 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:44.318383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 760/1524] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:45.443087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb55f-c2e1-4898-a367-8a56999ee3d5 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.443690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb55f-c2e1-4898-a367-8a56999ee3d5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.443690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79adb55f-c2e1-4898-a367-8a56999ee3d5 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.445055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 8 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 13:59:45.452698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690e8c-c556-4d65-bbb3-70ccf5041564 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.453486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690e8c-c556-4d65-bbb3-70ccf5041564 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.453486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8690e8c-c556-4d65-bbb3-70ccf5041564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.482518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c8690e8c-c556-4d65-bbb3-70ccf5041564 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:45.484027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c8690e8c-c556-4d65-bbb3-70ccf5041564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:45.495182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 8 13:59:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:45.651626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adb55f-c2e1-4898-a367-8a56999ee3d5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:45.675121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79adb55f-c2e1-4898-a367-8a56999ee3d5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:45.683450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:45.684834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:45.685156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:45.685517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:45.695470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:45.697309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8badcc1b-7811-4810-946f-de4137d7e12f service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:45.709580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 766/1527] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:49.905933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.906750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.906750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.908156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:49.914285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1a33-735d-432c-9d46-153749624713 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:49.914969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1a33-735d-432c-9d46-153749624713 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:49.915086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50a1a33-735d-432c-9d46-153749624713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:49.937504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a50a1a33-735d-432c-9d46-153749624713 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:49.938748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a50a1a33-735d-432c-9d46-153749624713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:49.947352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:59:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:50.244032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:50.245933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:50.276800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:50.310817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498928f1-fda6-4859-a01d-c4d59c5a5694 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:50.314529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:50.315185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:50.315491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:50.315760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:50.326755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:50.328606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869c2989-d7f0-450d-a5db-efaad0a9dfbb service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:50.339508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 763/1530] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:52.611241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619140b6-b5b7-4e12-b161-c2efaadf388c None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.611832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619140b6-b5b7-4e12-b161-c2efaadf388c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.611832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619140b6-b5b7-4e12-b161-c2efaadf388c None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.613244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:59:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:52.619235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be907a68-d880-4528-89bb-36642c6d9fe6 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.619769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be907a68-d880-4528-89bb-36642c6d9fe6 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.619769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be907a68-d880-4528-89bb-36642c6d9fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.641700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-be907a68-d880-4528-89bb-36642c6d9fe6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:52.642993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-be907a68-d880-4528-89bb-36642c6d9fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:52.651342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:59:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:52.950451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619140b6-b5b7-4e12-b161-c2efaadf388c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:52.961589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619140b6-b5b7-4e12-b161-c2efaadf388c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:52.967472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:52.968091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:52.968407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:52.968675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:52.978967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:52.980448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bfec40-a825-43b3-8a73-b3e9e04c49dd service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:52.990806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 769/1533] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:56.774563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:56.775146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:56.775146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:56.776521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:56.782453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e625-ed80-47af-b222-dcc429710fdd None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:56.783269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e625-ed80-47af-b222-dcc429710fdd None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:56.783269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28e625-ed80-47af-b222-dcc429710fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:56.790937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=74 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 13:59:56.804383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0b28e625-ed80-47af-b222-dcc429710fdd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:56.809905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0b28e625-ed80-47af-b222-dcc429710fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:56.815472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:59:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 13:59:57.129087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:57.130870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:57.197106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64577bc-b5a9-475a-8470-d7621ed632df None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:57.200730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:57.201346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:57.201630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:57.201905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:57.211879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:57.213450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e5921d-1383-4c86-9f3f-4ebbd86cce93 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:57.224752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 766/1536] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 13:59:59.572932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.573519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.573519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.575471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 8 13:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 13:59:59.581586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b049a-932f-4d39-ba88-3a5f33e09de8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.582160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b049a-932f-4d39-ba88-3a5f33e09de8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.582160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1b049a-932f-4d39-ba88-3a5f33e09de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.605449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0e1b049a-932f-4d39-ba88-3a5f33e09de8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 13:59:59.606758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0e1b049a-932f-4d39-ba88-3a5f33e09de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:59.615283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 8 13:59:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 13:59:59.929560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 13:59:59.931430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 13:59:59.952921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9cf01b-67e3-4453-a08f-b9cfbc1272be None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 13:59:59.957478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 13:59:59.958094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 13:59:59.958402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 13:59:59.958670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 13:59:59.969689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 13:59:59.971206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132607f-c3de-431f-a871-1513ff0799dd service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 13:59:59.981736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 772/1539] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 13:59:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:00:09.599835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=47 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:00:09.976893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=8 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:26.652454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:26.653104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:26.653387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:26.699143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:26.706699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:26.726849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 768/1540] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:26.750609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:26.751206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:26.751566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:26.806728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:26.809524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:26.832401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 773/1541] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:26.841663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:26.863657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b10279-4dfe-4138-b8bd-0964078014bb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:26.878960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:26.880065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:26.880564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:26.881059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:26.882268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:26.882801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:26.886028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:26.921109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 769/1542] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:26.938551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:26.961184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee500667-28bb-414c-bef1-2f0ffac575e2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:26.980982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:26.982213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:26.982809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:26.983293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:26.984494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:26.988125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:27.188175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 774/1543] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:26 2022] POST /identity/v3/projects => generated 393 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:27.194197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:27.206821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145d1f7f-d6a8-4ebd-a065-fb93dee73e6c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:27.291033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:27.291722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.292006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.292302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.293513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:27.295764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:27.443763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 770/1544] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:27 2022] POST /identity/v3/users => generated 297 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:27.450033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:27.480312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10672b91-1038-4a8e-ab4e-e1253bd41f23 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:27.491796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:27.492834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.493324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.493804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.495152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:27.500033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a9290-5943-461a-9e90-7f63efcf4a4f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:27.534976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 775/1545] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:27.541617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:27.542073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.542476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.543013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.576450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7e7617-acea-42f9-b936-2b1f925c5837 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:27.593024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:27.594577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.595372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.596285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.597569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:27.601479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd619f8-77e8-4e05-a1e6-a7f06599d2b7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:27.608185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:27.608898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:27.638542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 771/1546] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:27 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:27.642156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e979b6-7159-4984-9a23-c71f99fc7eb5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.643400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e979b6-7159-4984-9a23-c71f99fc7eb5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.648181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e979b6-7159-4984-9a23-c71f99fc7eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.648505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 776/1547] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:27.648505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:27.690647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb99700-08a8-4393-97dd-ae9a5dc792ca None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:27.713414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:27.714336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-49e979b6-7159-4984-9a23-c71f99fc7eb5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:27.715162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.715755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.716281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] PATH_INFO: `/v3/projects/44e7a5fc099c4041aa1a3109962b5dc3/users/a30b0734975d4e5a848be44aa7860807/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.718670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-49e979b6-7159-4984-9a23-c71f99fc7eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:27.748542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 772/1548] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:27.751852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07c5a-d442-4150-8579-79ab3a73b879 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.754860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07c5a-d442-4150-8579-79ab3a73b879 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.755678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c07c5a-d442-4150-8579-79ab3a73b879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.767719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e7a5fc099c4041aa1a3109962b5dc3, user_id=a30b0734975d4e5a848be44aa7860807, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:27.773410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:27.817398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c6c07c5a-d442-4150-8579-79ab3a73b879 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:27.820189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c6c07c5a-d442-4150-8579-79ab3a73b879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:27.844004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 773/1549] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:27.847174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98464d9-0f99-4c78-bd13-223bb6c4447a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:27.848183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98464d9-0f99-4c78-bd13-223bb6c4447a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:27.849003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98464d9-0f99-4c78-bd13-223bb6c4447a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:27.895147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 777/1550] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:27 2022] PUT /identity/v3/projects/44e7a5fc099c4041aa1a3109962b5dc3/users/a30b0734975d4e5a848be44aa7860807/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:27.896457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:27.905918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f98464d9-0f99-4c78-bd13-223bb6c4447a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:27.908537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f98464d9-0f99-4c78-bd13-223bb6c4447a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.095139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 774/1551] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.096636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6744fd0-470a-4fdb-af80-8c15e48eeb37 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.096943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6744fd0-470a-4fdb-af80-8c15e48eeb37 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.097233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6744fd0-470a-4fdb-af80-8c15e48eeb37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.135320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a316490-d3d2-438a-9d8f-c4d415b3a13c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.144999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.146056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.146688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.147203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.149004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:28.149511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c6744fd0-470a-4fdb-af80-8c15e48eeb37 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.149707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.150773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c6744fd0-470a-4fdb-af80-8c15e48eeb37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.151377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:28.151377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 14:01:28.151377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 14:01:28.156174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71c39f8-a9ca-4d66-9788-71affd395ad0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.178348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 775/1552] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.184260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.184691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.185177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.196814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 778/1553] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:28.200764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.208416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.208854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.254794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.263421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.277730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.282805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.294032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 776/1554] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.297961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.310281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 779/1555] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.311460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.339740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a13d10a-39ab-457c-98f1-d9020113508b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.349351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.350754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.351865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.352685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.354143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:28.355521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.358882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.379009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 777/1556] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:28.380238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.392907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73930463-5586-401f-8465-eb1b9c9fdf6d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.399111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.400095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.400654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.401148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.402022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.403909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:28.403909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 14:01:28.403909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 14:01:28.405544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7dec16-b5e9-4958-a8a6-edd7ebb5aea3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.409929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d19611-495a-4c35-9c4d-7ebfdd53a86c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.417451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.417451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.417451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.417451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.439306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.440977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33df310-fb40-4055-b297-f4bfa6e2985a service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.453765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 780/1557] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:28.464887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a398-f2d3-4a53-a7f6-6d3291c464d0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.468929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a398-f2d3-4a53-a7f6-6d3291c464d0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.468929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a11a398-f2d3-4a53-a7f6-6d3291c464d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.502384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4a11a398-f2d3-4a53-a7f6-6d3291c464d0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.503893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4a11a398-f2d3-4a53-a7f6-6d3291c464d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.513652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 781/1558] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.515100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.515486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.515886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.540237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:28.551247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.554069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.554505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 778/1559] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/projects => generated 367 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:28.560707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.560707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.560707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.570008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:28.583980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.586601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.652276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 782/1560] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.653314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.670807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6240d939-711e-42cc-ba56-69a48bf61380 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.678828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.685049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 779/1561] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.685359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.685359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.685359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.686128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.686288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.686288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.695215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f9a2c4-4c21-4ff4-bfd8-2e9f5faeec56 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.700957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.701685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.701968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.702241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.702978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:28.703281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.705030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9f3c79-e68b-4392-92be-663a8a3d4516 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.723136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 780/1562] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:28.725476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636c632-4891-4a4d-b75c-3042dd904b7a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.725980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636c632-4891-4a4d-b75c-3042dd904b7a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.726475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b636c632-4891-4a4d-b75c-3042dd904b7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.759821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b636c632-4891-4a4d-b75c-3042dd904b7a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.762598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b636c632-4891-4a4d-b75c-3042dd904b7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.776309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 781/1563] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.780580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a499-348e-4a1d-a95e-0c0e371e7b91 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.780580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a499-348e-4a1d-a95e-0c0e371e7b91 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.780580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2a499-348e-4a1d-a95e-0c0e371e7b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.814097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-cea2a499-348e-4a1d-a95e-0c0e371e7b91 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.816246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cea2a499-348e-4a1d-a95e-0c0e371e7b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.832038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 783/1564] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:28.832038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.833056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 782/1565] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.833056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.833056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.833473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.866429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:28.868382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:28.909333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe86f69-ee28-4d07-b337-e12c3ff339d2 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.915518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.916321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.916667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:28.917038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:28.917716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:28.918992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:28.918992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 14:01:28.918992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 14:01:28.920654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5af5a07-0795-4747-ae69-2207fabb7064 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:28.974090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 783/1566] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:28.975200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:28.991582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ce04b-3dea-4657-8a41-537a515989a0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:28.999266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:28.999266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:28.999485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.000092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.001091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:29.001091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.002816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.020736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 784/1567] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:29.022061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.041346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6897e1-3ac1-4a50-97f9-0f1448523594 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.047280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.047893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.048192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.048467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.049109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.051004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba6e641-af5e-4459-abe1-dcb2200d4487 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.099153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 784/1568] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:28 2022] POST /identity/v3/users => generated 284 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.100791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.101138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.102823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.124444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:29.125701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:29.135795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 785/1569] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:29.136526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.148003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f673347-9786-47a6-94b8-e7deff88efe0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.154422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.154422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.154422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.154422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.154976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.156966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.205518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 785/1570] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/projects => generated 409 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.207917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017868-c562-4999-8292-c9292f9a259a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.208481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017868-c562-4999-8292-c9292f9a259a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.209010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30017868-c562-4999-8292-c9292f9a259a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.230161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-30017868-c562-4999-8292-c9292f9a259a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:29.231887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-30017868-c562-4999-8292-c9292f9a259a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:29.282712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 786/1571] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/users => generated 289 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.286909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.289354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:29.298407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7613e3f-9c40-44d3-9825-bea1ffd022f2 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.339328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 786/1572] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:29.339328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30017868-c562-4999-8292-c9292f9a259a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.349489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30017868-c562-4999-8292-c9292f9a259a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.354875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.356198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.356567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.356919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.357683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.359981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.374042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:29.381401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.382235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.382553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.382839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.383487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:29.384244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.386003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.404191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 787/1573] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:29.404191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.421207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeb94f7-1acf-4bb2-bc4f-1d69a4c0258b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.426457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.427079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.427359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.427626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.428429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.430063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.443064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 788/1574] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:29.444281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.456664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fd083a-15ac-4294-bd11-4ad362c99b81 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.464047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.464865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.465233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.465614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.466760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:29.467203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.469372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.489615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 789/1575] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:29.490962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.512512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81189abf-25f1-429a-a8e4-ea87a1feb268 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.520555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.528766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.528766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.528766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.528766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.528766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.597645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 787/1576] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.598500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.637227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 790/1577] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/users => generated 305 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.641631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.661777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e8baf-7a62-4ab5-b111-bc0e7735be3e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.682934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa612234-4cd1-4b4f-a828-b293f266f9a7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.689158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.690226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.690538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.690904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.691679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.693714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.706908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 788/1578] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:29.707752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.722286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e69e7-d2cb-44d6-9d2b-abb4cac2ad04 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.728766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.730559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.730559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.730559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.731220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.734117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.771289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.772246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.772772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.773313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.774440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:29.774976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.777602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.795083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 791/1579] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:29.795083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.807526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c148d50-25de-4d90-a24b-4e65077b881a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.813861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.817534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.817534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.817534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.817534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.818232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.831071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 792/1580] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:29.831929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.842526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d421c2-92c5-4674-a782-dcd102e10aa7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.847585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.848362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.848707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.849037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.849745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.851779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.862547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 789/1581] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/projects => generated 365 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.862547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:29.938404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f32bf7-4dff-49a3-acf8-1256544b7e08 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:29.944199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:29.949822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:29.949822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:29.949822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:29.949822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:29.949822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:29.987605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 793/1582] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/projects => generated 379 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:29.989568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.065142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068918a7-eb6e-4b86-a7e3-663c669a2ffc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.072050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.073353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.073353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.073712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.074578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.076664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.090387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 794/1583] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:29 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:30.091595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.105150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a73602-c783-4a3a-9949-9e9cde70360a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.112275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.113010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.113389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.113722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.114491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.116607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.118712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:30.148236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 795/1584] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:30.148236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.149066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 790/1585] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:29 2022] POST /identity/v3/users => generated 287 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:30.149066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.165338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea71c07-8c64-4365-b43a-81100e5d6c31 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.171907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33bbae9-6ec8-4076-900d-e5161e1156d6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.173454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.174211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.174482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.174926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.175265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.175443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.175708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.176168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] PATH_INFO: `/v3/projects/8dd4951d620c4bf4b5b0ba9f257424e3/users/ce143361f42046d29e6b191037331fbc/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.176440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.179876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.217661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8dd4951d620c4bf4b5b0ba9f257424e3, user_id=ce143361f42046d29e6b191037331fbc, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.218725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:30.218725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 14:01:30.218725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 14:01:30.220855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.224732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:30.320060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 791/1586] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:30 2022] PUT /identity/v3/projects/8dd4951d620c4bf4b5b0ba9f257424e3/users/ce143361f42046d29e6b191037331fbc/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:30.321027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.328743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 796/1587] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:30 2022] POST /identity/v3/projects => generated 385 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:30.330469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.397784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a817cf8e-3f9d-4510-91f1-1ae8141ceca1 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.450821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4925384-6928-4f87-acb9-371b11830b30 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.459198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.459948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.460270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.460545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.461193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.463344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.479591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.480319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.480713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.480981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.481670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.483674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.576791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 797/1588] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:30 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:30.578026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.589883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afcc20a-b822-4351-9196-5d49d2b0277c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.595990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.599851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.600317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.600597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.601341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.603035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.618854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 792/1589] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:30 2022] POST /identity/v3/users => generated 283 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:30.620157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.626365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 798/1590] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:30.630188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.638858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2be1a4-87e7-4cd5-b3f8-d63aa6175763 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.642315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f2b541-8455-48fe-bc57-553f45d89f8b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.646890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.657807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.657807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.658170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.658362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] PATH_INFO: `/v3/projects/73ba0ac1c50e4802b8c055a9b3075022/users/c18dc6b5898c4e1aae71a028b4840bf0/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.658552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.658552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.658552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.658552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.658552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.668464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 799/1591] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:30.668464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.676674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73ba0ac1c50e4802b8c055a9b3075022, user_id=c18dc6b5898c4e1aae71a028b4840bf0, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.680273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55718ad1-8194-4315-84bf-72124b58ace9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.682143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186fc1a0-7752-4034-8bae-30c7cb00db58 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.689064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:30.689934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.690367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.690808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.691582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:30.694130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:30.800974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 793/1592] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:30 2022] PUT /identity/v3/projects/73ba0ac1c50e4802b8c055a9b3075022/users/c18dc6b5898c4e1aae71a028b4840bf0/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:30.800974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:30.800974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:30.801890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:30.847586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] MFA Rules not processed for user `ce143361f42046d29e6b191037331fbc`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:30.848910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:30.892332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 800/1593] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:30 2022] POST /identity/v3/users => generated 293 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:30.893506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:30.914017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8739208c-a842-4263-9b75-61ebc2e941bc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:30.949792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:30.956085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 794/1594] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:30.963779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.044051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5591a2-e5ae-40a2-87ad-ade80239bf50 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.048867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.049899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.050661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.050934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.051293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.051637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.052226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.052622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.053187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.054584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.054863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.059488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.076565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 795/1595] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.077646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.085983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 801/1596] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.088403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.094006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28990b4-85d0-44bb-aee5-1ca5dc471fe5 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.100071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.101039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.101535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.102166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] PATH_INFO: `/v3/projects/769f033e409344a18a3e74c7b9314caa/users/6f87715ef916491e889c3b9f7322fc19/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.113897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435a1abf-a4d8-4792-86fa-b86286332ead admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.124404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.124701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=769f033e409344a18a3e74c7b9314caa, user_id=6f87715ef916491e889c3b9f7322fc19, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.125671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.126179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.126651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.127486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.127752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.130593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfae4983-3e56-40db-821c-d5dd300a2f8c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.145359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 802/1597] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.148221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.148850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.153298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.170873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:31.172109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:31.181006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 803/1598] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:31.187001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.195752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:31.203093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13944304-eb40-4ba9-8db2-789e8b90c87a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.209354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.210154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.210452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.210730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.211498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.213184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.232874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 804/1599] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.234076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.328676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33062adf-e3cb-4866-8bd6-d049b7d45946 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.333050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 796/1600] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:31 2022] PUT /identity/v3/projects/769f033e409344a18a3e74c7b9314caa/users/6f87715ef916491e889c3b9f7322fc19/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:31.343600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.478361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.478742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.478933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.479471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.520267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd686c-092a-4f51-b446-5f2876fc6339 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.529731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.530779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.531246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.531701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.532936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.535904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.563776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 797/1601] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.565897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.586281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034ec86d-dc97-45ef-9c58-d4c118881952 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.597137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.598134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.598611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.599116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.600483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.603218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.607272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.610522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.624757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 798/1602] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:31.625854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.634603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 805/1603] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:31.636367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.639341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:31.643861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9e19c-1b2c-480c-b629-737b00b71d32 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.655919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.657083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.657630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.658034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] PATH_INFO: `/v3/projects/95d4d80d60924866ab0240d1b4f251cf/users/c5bfdc5f93e740229e2558df1a855d29/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.687030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95d4d80d60924866ab0240d1b4f251cf, user_id=c5bfdc5f93e740229e2558df1a855d29, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.694694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.769382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6282b4-ee65-46dd-85b9-86b57bfbccb1 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:31.784373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:31.784373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.784373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.784373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.802157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:31.804737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88369ae9-94e3-4920-8e4e-64162b1ae86d service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:31.828208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 806/1604] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:31.836957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 799/1605] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:31 2022] PUT /identity/v3/projects/95d4d80d60924866ab0240d1b4f251cf/users/c5bfdc5f93e740229e2558df1a855d29/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:31.839427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:31.849335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:31.849888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:31.849888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:31.869510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:31.917009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] MFA Rules not processed for user `6f87715ef916491e889c3b9f7322fc19`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:31.920280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:31.957662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:32.007077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb66bd-176c-4167-b698-58e612226854 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.012877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.019425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.019425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.019425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.019425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.019425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.033274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 800/1606] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:32.034378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:32.048838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848f9e0-21d0-4099-8adf-264e86529ea6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.054977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.055729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.056032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.056321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] PATH_INFO: `/v3/projects/43b3aef29e6d4d3b9efa5cdd9e47aa6d/users/1064ceb4bf0d4b6f82f86b22e1ca7e63/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.225492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b3aef29e6d4d3b9efa5cdd9e47aa6d, user_id=1064ceb4bf0d4b6f82f86b22e1ca7e63, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.225492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afc7dbe-9968-40fe-a005-5d8aef2ac826 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.226496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 807/1607] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:31 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:32.226496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:32.226496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c055f3f-adbb-417e-b111-4488a7665cec None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.226496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.227104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.227104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.227104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] PATH_INFO: `/v3/projects/9cfcfe1ebb994061b6d203510fd7e65b/users/c96336eb4bfd461e82e7761077a5b910/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.236494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cfcfe1ebb994061b6d203510fd7e65b, user_id=c96336eb4bfd461e82e7761077a5b910, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.242017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.295952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 801/1608] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:32 2022] PUT /identity/v3/projects/43b3aef29e6d4d3b9efa5cdd9e47aa6d/users/1064ceb4bf0d4b6f82f86b22e1ca7e63/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:32.304624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.304624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.304624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.377133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] MFA Rules not processed for user `c5bfdc5f93e740229e2558df1a855d29`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:32.765069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:32.767792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 808/1609] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:32 2022] PUT /identity/v3/projects/9cfcfe1ebb994061b6d203510fd7e65b/users/c96336eb4bfd461e82e7761077a5b910/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:32.767792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:32.771228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 802/1610] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:32.771228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:32.772109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d3b7a-b6d3-4e88-a859-955292dd9949 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.772109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.772109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.772109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.775829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] PATH_INFO: `/v3/projects/92a3e908b87b445b9674d48f9a61bf94/users/9b0569d9de724d5fbe3f41cbc0687d75/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.775829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92a3e908b87b445b9674d48f9a61bf94, user_id=9b0569d9de724d5fbe3f41cbc0687d75, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.775829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78d6c43-a478-46f6-bf11-42b3e8d48590 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.776864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec8397-c9f6-4aaa-bcd6-c99f0886168a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:32.776864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:32.776864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.776864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.777450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.777450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:32.777450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1279f9-c8be-4e2b-8986-a6679ab759ab admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:32.937876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 809/1611] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:32 2022] PUT /identity/v3/projects/92a3e908b87b445b9674d48f9a61bf94/users/9b0569d9de724d5fbe3f41cbc0687d75/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:32.939231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 803/1612] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:32 2022] POST /identity/v3/projects => generated 409 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:32.941219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.941730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.942204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.948424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:32.948424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:32.948424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:32.986485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:32.988927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.020390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:33.021141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] MFA Rules not processed for user `1064ceb4bf0d4b6f82f86b22e1ca7e63`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:33.022462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.110728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:33.121734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 804/1613] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:33.123605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.143756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847abfed-2e06-40a0-ac3c-3d54657588ab None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.146669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 810/1614] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:33.149140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.157484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.158372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.158905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.159470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.160363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.162497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.169388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edec015-ff96-4a19-bd0c-9297bac29e94 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.176369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.177546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.177935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.178310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.179872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.184526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1296874e-81b8-434f-9653-0221ce69ba4f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.407807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 811/1615] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/users => generated 311 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:33.414679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.421320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.422473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 805/1616] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:33.422820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.426044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.474720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:33.477731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:33.554007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a183a-8633-4166-ad05-94aec11ca067 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.577586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.580403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.580403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.580403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.580403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.582570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.644333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 812/1617] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:33.645485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.726134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 806/1618] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/projects => generated 363 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:33.726134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.796985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62360aa3-85b8-4c05-a066-616860c5ab37 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.894186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d74ebf7-56ab-4906-8bc1-edd864fb7ba0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.905763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.906826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.907297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.907783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.909043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.912247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.922533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.924826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.925344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.925967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.940105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 807/1619] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:33 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:33.942669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:33.962081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.962564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761fd364-c679-4276-84f0-574cd09f947a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:33.967528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.974919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:33.977741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:33.989587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:33.989587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:33.989587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:33.989587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:33.993187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 813/1620] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:33.994277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.006764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8a679-f2da-4f47-9455-9c71cff37bcf service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.013338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.014043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.014844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.015123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.030304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.034017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.054096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 814/1621] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:34.055236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.086380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051de1cc-926a-4e1c-b5d7-e1f3675a3680 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.086380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.086380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.086380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.087051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.087051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.087368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.179856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 808/1622] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:33 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:34.183220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.202786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b6a8b7-ee61-47a8-b04d-2d04229c10b9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.212509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.213987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.215207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.215811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] PATH_INFO: `/v3/projects/493f7b105deb418db42b396ffe7454c3/users/74f0118986d94e1d8c829effd35cf3a7/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.220780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:34.225278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 815/1623] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:34 2022] POST /identity/v3/users => generated 288 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:34.226707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.243899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1977bb-89ea-4c3c-94e5-eb3e62dc0359 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.252343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493f7b105deb418db42b396ffe7454c3, user_id=74f0118986d94e1d8c829effd35cf3a7, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.252596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.252953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.253130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.253405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.254061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.255741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.258931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.285336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 816/1624] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:34.287452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.299084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993851d-8820-4270-80ce-221d5301cb59 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.304348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.305166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.305516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.305860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.306782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.309350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.330588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 817/1625] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:34.331636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.351236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:34.352900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53181c3-c567-4a54-ab73-635700dd83dd admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.360833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.362175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.362792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.363303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] PATH_INFO: `/v3/projects/6df2770c5b254296b571665775b97a5b/users/df8539475c524d44a7f24a93a1ab99b3/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.385338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df2770c5b254296b571665775b97a5b, user_id=df8539475c524d44a7f24a93a1ab99b3, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.389584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.399857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 809/1626] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:34 2022] PUT /identity/v3/projects/493f7b105deb418db42b396ffe7454c3/users/74f0118986d94e1d8c829effd35cf3a7/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:34.401020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.507001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8b3d1f-8ded-44c7-ab66-b879afc37640 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.517175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.518399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.519057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.519550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] PATH_INFO: `/v3/projects/704e0d3c9b47411babb58e57ba7cb00b/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.551082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=704e0d3c9b47411babb58e57ba7cb00b, user_id=0a8f6d8b87ab44e0b2b9a5afdd66f7f2, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.556495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.587669 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 818/1627] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:34 2022] PUT /identity/v3/projects/6df2770c5b254296b571665775b97a5b/users/df8539475c524d44a7f24a93a1ab99b3/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:34.592451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.702453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a118bf-de93-4646-9b89-c7f6740229d1 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.720045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.723243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.723925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.724257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.725176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.729837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 810/1628] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:34 2022] PUT /identity/v3/projects/704e0d3c9b47411babb58e57ba7cb00b/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:34.734694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.736021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.783372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 819/1629] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:34.783372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:34.807575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e64e76-077f-444d-b216-3678f1915d55 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.911244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840c0c9-f25d-4c1f-8c3e-febb951007fc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:34.941388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:34.942565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:34.951296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:34.951296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:34.951296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:34.951296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:34.972350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 811/1630] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:34.974326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.023930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e1852-9480-446c-bcfd-bac00f0d1ccf admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.075237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.083122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.085173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.087243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] PATH_INFO: `/v3/projects/493f7b105deb418db42b396ffe7454c3/users/74f0118986d94e1d8c829effd35cf3a7/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.088319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.091324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.091792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.095535 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.100055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.102056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=493f7b105deb418db42b396ffe7454c3, user_id=74f0118986d94e1d8c829effd35cf3a7, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.109700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.134340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.155929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 820/1631] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:35.155929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.183353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fd4f37-aeb2-4dd0-a6fb-ce826a339f19 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.199779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 812/1632] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:34 2022] PUT /identity/v3/projects/493f7b105deb418db42b396ffe7454c3/users/74f0118986d94e1d8c829effd35cf3a7/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:35.199779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.278522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.281526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.281856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.282084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] PATH_INFO: `/v3/projects/6df2770c5b254296b571665775b97a5b/users/df8539475c524d44a7f24a93a1ab99b3/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.284867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6df2770c5b254296b571665775b97a5b, user_id=df8539475c524d44a7f24a93a1ab99b3, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.307487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a90f793-c662-405e-9ea8-c3605c2b4b4a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.344595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 821/1633] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:35 2022] PUT /identity/v3/projects/6df2770c5b254296b571665775b97a5b/users/df8539475c524d44a7f24a93a1ab99b3/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:35.359079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7536f-1069-4f91-86ec-5d88c8b1d824 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.362085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7536f-1069-4f91-86ec-5d88c8b1d824 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.362414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7536f-1069-4f91-86ec-5d88c8b1d824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.382051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51395718-e183-41ff-b4f9-2ebaf7905b93 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:35.488582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dea7536f-1069-4f91-86ec-5d88c8b1d824 None None] MFA Rules not processed for user `74f0118986d94e1d8c829effd35cf3a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.491432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dea7536f-1069-4f91-86ec-5d88c8b1d824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.533042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:35.533889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.535521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.535920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] PATH_INFO: `/v3/projects/704e0d3c9b47411babb58e57ba7cb00b/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.541506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] RBAC: Authorizing `identity:create_grant(project_id=704e0d3c9b47411babb58e57ba7cb00b, user_id=0a8f6d8b87ab44e0b2b9a5afdd66f7f2, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:35.549349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc33fc-5c7b-47b5-b80c-5acbc953d497 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:35.598377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 813/1634] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:35 2022] PUT /identity/v3/projects/704e0d3c9b47411babb58e57ba7cb00b/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 398 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:35.600119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.600563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.600948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.661238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 822/1635] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:35.663156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:35.663632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:35.664122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:35.667181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] MFA Rules not processed for user `df8539475c524d44a7f24a93a1ab99b3`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.671220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.736090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] MFA Rules not processed for user `0a8f6d8b87ab44e0b2b9a5afdd66f7f2`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:35.737754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:35.870183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 814/1636] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:35.871945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.942474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 823/1637] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:35 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:35.946095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:35.997889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a53d8-63a7-4cfa-8a59-4a8684f17519 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.010896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.012815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.013104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.014825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.181190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.185144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.208166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 815/1638] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:35 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:36.225418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a50bdde-28c8-4028-95a8-df1e9dff79c5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.234046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.235979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.237233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.237951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.275751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.278133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633da322-b2a8-4eb1-a709-a9336bcf34fd service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.326120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 824/1639] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:36.344868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:36.357541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82932c6-71d5-4cc1-b02f-8cd75ba2c2f6 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:36.369151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:36.369799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.370084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.370363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.406664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:36.414333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce3e0ac-6611-4414-940c-b2ab794692db service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:36.441151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 816/1640] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:01:36.586867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54633-5236-4b46-9c00-472e3d18e924 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.587558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54633-5236-4b46-9c00-472e3d18e924 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.588244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a54633-5236-4b46-9c00-472e3d18e924 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.596667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 825/1641] 10.222.0.97 () {54 vars in 962 bytes} [Thu Sep 8 14:01:36 2022] GET /identity => generated 270 bytes in 27 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:01:36.612330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8851-d5bb-4064-b579-0a654f093571 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.612823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8851-d5bb-4064-b579-0a654f093571 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.613218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8851-d5bb-4064-b579-0a654f093571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.724452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-fb5a8851-d5bb-4064-b579-0a654f093571 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:36.727036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fb5a8851-d5bb-4064-b579-0a654f093571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:36.740315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:36.920601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 817/1642] 10.222.0.97 () {58 vars in 1059 bytes} [Thu Sep 8 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:36.929397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02672-e32b-45b2-bdfa-7d92813a5966 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.929889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02672-e32b-45b2-bdfa-7d92813a5966 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.930361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb02672-e32b-45b2-bdfa-7d92813a5966 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.933302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 826/1643] 10.222.0.97 () {54 vars in 912 bytes} [Thu Sep 8 14:01:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 14:01:36.944213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:36.944651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:36.945041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:36.991350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:36.991350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:37.029762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 818/1644] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:01:36 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:37.192447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:37.462187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02672-e32b-45b2-bdfa-7d92813a5966 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.501718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb02672-e32b-45b2-bdfa-7d92813a5966 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.514791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.519270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.519882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.520359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.571709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:37.582779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.585375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.607205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12f38bb-4beb-4901-a841-4a5aadbe846f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:37.619506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:37.619809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 827/1645] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:37.621371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:37.624068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:37.624630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:37.821773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:37.823385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:37.858351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 819/1646] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:37 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:40.200000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:40.661175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:41.671495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:41.700107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:41.876676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1bcd6-c59b-49f3-b636-9778119c5bdd service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:41.901511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:41.905972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:41.906302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:41.907184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] PATH_INFO: `/v3/users/c5bfdc5f93e740229e2558df1a855d29` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:41.914059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5bfdc5f93e740229e2558df1a855d29)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:41.922284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:41.941921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81654fb-612e-492f-9f6a-a641a04de91a service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:41.955569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:41.957862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:41.958171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:41.960011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] PATH_INFO: `/v3/users/1064ceb4bf0d4b6f82f86b22e1ca7e63` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:41.965543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1064ceb4bf0d4b6f82f86b22e1ca7e63)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:41.966962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: WARNING py.warnings [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:41.966962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: warnings.warn(msg) Sep 08 14:01:41.966962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: Sep 08 14:01:41.972358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:42.319602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:42.410316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5bfdc5f93e740229e2558df1a855d29'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.411601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5bfdc5f93e740229e2558df1a855d29'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.417745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1064ceb4bf0d4b6f82f86b22e1ca7e63'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.423711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1064ceb4bf0d4b6f82f86b22e1ca7e63'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.442935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5bfdc5f93e740229e2558df1a855d29'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.446449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5bfdc5f93e740229e2558df1a855d29'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.469035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1064ceb4bf0d4b6f82f86b22e1ca7e63'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.502005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d98944d4-420e-4961-b5da-e286a377fb70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5bfdc5f93e740229e2558df1a855d29'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.547626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1064ceb4bf0d4b6f82f86b22e1ca7e63'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.552927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2652aa27-d389-44a1-afdb-3b75c02f0b8a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1064ceb4bf0d4b6f82f86b22e1ca7e63'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:42.795052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 828/1647] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:41 2022] DELETE /identity/v3/users/c5bfdc5f93e740229e2558df1a855d29 => generated 0 bytes in 1125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:42.798436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567be4-3812-47f3-8874-0bfbad4c8ea6 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:42.801713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567be4-3812-47f3-8874-0bfbad4c8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:42.802458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf567be4-3812-47f3-8874-0bfbad4c8ea6 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:42.807954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 829/1648] 10.222.0.97 () {54 vars in 962 bytes} [Thu Sep 8 14:01:42 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:01:42.811027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3ce9-5e98-4642-b9b6-b76095727b5e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:42.811969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3ce9-5e98-4642-b9b6-b76095727b5e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:42.812701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d3ce9-5e98-4642-b9b6-b76095727b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:42.880884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-068d3ce9-5e98-4642-b9b6-b76095727b5e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:42.884730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-068d3ce9-5e98-4642-b9b6-b76095727b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.011168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 820/1649] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:41 2022] DELETE /identity/v3/users/1064ceb4bf0d4b6f82f86b22e1ca7e63 => generated 0 bytes in 1313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:43.014092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237ca8b-7b4b-48e6-aa5c-c0b25af7ed20 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.015100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237ca8b-7b4b-48e6-aa5c-c0b25af7ed20 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.017757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237ca8b-7b4b-48e6-aa5c-c0b25af7ed20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.055413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:43.077231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1237ca8b-7b4b-48e6-aa5c-c0b25af7ed20 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:43.081959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1237ca8b-7b4b-48e6-aa5c-c0b25af7ed20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.098705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 830/1650] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:43.101355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.101947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.102599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.152444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:43.154688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.296441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 821/1651] 10.222.0.97 () {58 vars in 1059 bytes} [Thu Sep 8 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:43.306734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c3790-246b-4b3c-a5a2-5d223ac812bc None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.307422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c3790-246b-4b3c-a5a2-5d223ac812bc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.307965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03c3790-246b-4b3c-a5a2-5d223ac812bc None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.309768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 822/1652] 10.222.0.97 () {54 vars in 912 bytes} [Thu Sep 8 14:01:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:01:43.339021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.340360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.340816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.388416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:43.389700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.412330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 823/1653] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 14:01:43.434055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:43.435529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.486047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 831/1654] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:43.487191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:43.490874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:43.657929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:43.682595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1b7adb-86cc-46bd-9cc0-2d4d48bbd46f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:43.713182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:43.714258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.715044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.738311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.780942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:43.781953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1614256b-9371-47ba-88b7-1cafa098d98e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:43.785382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:43.797060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:43.798166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.798509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.798787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.828401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:43.829611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 824/1655] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:43.830273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba21cd9-bc70-4b8b-acd6-4f430f448b89 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:43.833228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:43.865376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b73ce9-310c-4bde-9db6-f87371c95d82 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:43.865770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 832/1656] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:43.876066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:43.878092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:43.879117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:43.879570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:43.952499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:43.975149 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ead6ab-7ae9-447a-b106-3e9b4e1e64f6 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:44.033969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 825/1657] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:44.995551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:44.996934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:44.996934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.078252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] MFA Rules not processed for user `a30b0734975d4e5a848be44aa7860807`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:45.080172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.280799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 833/1658] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:45.284536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:45.285197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:45.285613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.329251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] MFA Rules not processed for user `a30b0734975d4e5a848be44aa7860807`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:45.335406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.361593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 826/1659] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:45 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:45.393958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:45.396209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.461811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450b9b8-4c2c-48e2-a00f-18dc3932e420 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:45.477745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:45.479079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:45.479769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:45.480525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.515966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:45.526943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:45.566755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 834/1660] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:45.880602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:45.883188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:45.934087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f47ab81-7ce5-4d93-bd96-80d466e484da None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:45.948820 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:45.949581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:45.949882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:45.950196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] PATH_INFO: `/v3/projects/43b3aef29e6d4d3b9efa5cdd9e47aa6d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:45.951805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b3aef29e6d4d3b9efa5cdd9e47aa6d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:45.953862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:45.976717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:45.978257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:46.006346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7a31aa-ea32-44b3-8a4b-e523979354f6 service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:46.016716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:46.017763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:46.018186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:46.019212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] PATH_INFO: `/v3/projects/95d4d80d60924866ab0240d1b4f251cf` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:46.020932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95d4d80d60924866ab0240d1b4f251cf)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:46.022362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: WARNING py.warnings [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 08 14:01:46.022362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: warnings.warn(msg) Sep 08 14:01:46.022362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: Sep 08 14:01:46.024257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:46.220432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b3aef29e6d4d3b9efa5cdd9e47aa6d', 'request_id': 'req-3fa81f40-25ee-4302-85d7-1926c9d148ec'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.256528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b3aef29e6d4d3b9efa5cdd9e47aa6d', 'request_id': 'req-3fa81f40-25ee-4302-85d7-1926c9d148ec'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.287886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b3aef29e6d4d3b9efa5cdd9e47aa6d', 'request_id': 'req-3fa81f40-25ee-4302-85d7-1926c9d148ec'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.304403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95d4d80d60924866ab0240d1b4f251cf', 'request_id': 'req-d322fd00-2508-4423-9c8e-ee4593cc8307'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.349968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95d4d80d60924866ab0240d1b4f251cf', 'request_id': 'req-d322fd00-2508-4423-9c8e-ee4593cc8307'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.368530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95d4d80d60924866ab0240d1b4f251cf', 'request_id': 'req-d322fd00-2508-4423-9c8e-ee4593cc8307'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.441638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] The token cache is being invalidate because project 43b3aef29e6d4d3b9efa5cdd9e47aa6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:46.443462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.444364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.450696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 827/1661] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:45 2022] DELETE /identity/v3/projects/43b3aef29e6d4d3b9efa5cdd9e47aa6d => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:46.452478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:46.457006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:46.579580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] The token cache is being invalidate because project 95d4d80d60924866ab0240d1b4f251cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:46.580123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.586174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:46.588744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 835/1662] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:45 2022] DELETE /identity/v3/projects/95d4d80d60924866ab0240d1b4f251cf => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:46.594659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:46.596373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:46.713560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa81f40-25ee-4302-85d7-1926c9d148ec admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:46.853497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d322fd00-2508-4423-9c8e-ee4593cc8307 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:46.876037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:46.877816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:46.878125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:46.878406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] PATH_INFO: `/v3/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:46.880777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:46.882278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a8f6d8b87ab44e0b2b9a5afdd66f7f2)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:46.882544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:46.883478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:46.884300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] PATH_INFO: `/v3/users/df8539475c524d44a7f24a93a1ab99b3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:46.884608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:46.887113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df8539475c524d44a7f24a93a1ab99b3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:46.891410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:47.229767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a8f6d8b87ab44e0b2b9a5afdd66f7f2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.231750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a8f6d8b87ab44e0b2b9a5afdd66f7f2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.244781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a8f6d8b87ab44e0b2b9a5afdd66f7f2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.264392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a8f6d8b87ab44e0b2b9a5afdd66f7f2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.265497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e7aabb-0a3f-4b20-a40f-09399c3be2e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a8f6d8b87ab44e0b2b9a5afdd66f7f2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.278344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:47.346906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df8539475c524d44a7f24a93a1ab99b3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.347973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df8539475c524d44a7f24a93a1ab99b3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.360444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df8539475c524d44a7f24a93a1ab99b3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.361804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df8539475c524d44a7f24a93a1ab99b3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.385053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df8539475c524d44a7f24a93a1ab99b3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:47.465960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 828/1663] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:46 2022] DELETE /identity/v3/users/0a8f6d8b87ab44e0b2b9a5afdd66f7f2 => generated 0 bytes in 1014 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:47.468251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde29c5f-30bc-4f05-b854-e42285f22fb0 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:47.468710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde29c5f-30bc-4f05-b854-e42285f22fb0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:47.469119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde29c5f-30bc-4f05-b854-e42285f22fb0 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:47.470672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 829/1664] 10.222.0.97 () {54 vars in 912 bytes} [Thu Sep 8 14:01:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:01:47.482140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:47.484423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:47.484958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:47.635629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:47.641826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:47.674693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 836/1665] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:46 2022] DELETE /identity/v3/users/df8539475c524d44a7f24a93a1ab99b3 => generated 0 bytes in 1070 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:01:47.901473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 830/1666] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:01:47 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:48.018292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:48.020879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:48.045170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:48.235982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11467121-42b5-47d5-8b8b-11bbbaab81a3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:48.260599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:48.261405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:48.262326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:48.262704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:48.285424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:48.291160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:48.318907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 837/1667] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:48.870911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:49.249634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:49.254213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:49.576392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecfafbd-5242-4c34-baf4-613c54cdf7ad None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:49.966219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:49.971116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:49.973064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:49.973348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] PATH_INFO: `/v3/projects/6df2770c5b254296b571665775b97a5b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:49.986460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6df2770c5b254296b571665775b97a5b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:50.027846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:50.372298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:50.372298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:50.507071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6df2770c5b254296b571665775b97a5b', 'request_id': 'req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.534212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6df2770c5b254296b571665775b97a5b', 'request_id': 'req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.574767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6df2770c5b254296b571665775b97a5b', 'request_id': 'req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.576027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eedb66-8c87-4ecd-afd5-181a803a83ad service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:50.750926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] The token cache is being invalidate because project 6df2770c5b254296b571665775b97a5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:50.751320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.755351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-baa5d922-1b7c-4e36-b82f-4ed25b4f3c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:50.764988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:50.766122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:50.766536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 831/1668] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:49 2022] DELETE /identity/v3/projects/6df2770c5b254296b571665775b97a5b => generated 0 bytes in 1511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:50.766693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:50.768066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55015-bfc8-4bb2-b504-cc3c46997fa9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:50.768344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:50.768488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55015-bfc8-4bb2-b504-cc3c46997fa9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:50.768661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff55015-bfc8-4bb2-b504-cc3c46997fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:50.775003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:50.793662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:50.795475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b1f8a8-c6ed-470b-afac-a95c685d783d service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:50.817457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4ff55015-bfc8-4bb2-b504-cc3c46997fa9 None None] MFA Rules not processed for user `a30b0734975d4e5a848be44aa7860807`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:50.819590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4ff55015-bfc8-4bb2-b504-cc3c46997fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:50.821276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 838/1669] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:50 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:50.823060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa356-efb5-44ae-a798-ce80f50158c1 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:50.823367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa356-efb5-44ae-a798-ce80f50158c1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:50.823647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48fa356-efb5-44ae-a798-ce80f50158c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:50.888888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b48fa356-efb5-44ae-a798-ce80f50158c1 None None] MFA Rules not processed for user `c96336eb4bfd461e82e7761077a5b910`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:50.890183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b48fa356-efb5-44ae-a798-ce80f50158c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.099448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 832/1670] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:51.116658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.116946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.121660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.182532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 839/1671] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:50 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:51.182706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:51.183408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.199730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.200165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.200591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.297037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] MFA Rules not processed for user `c96336eb4bfd461e82e7761077a5b910`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:51.299626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.343206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 840/1672] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:51.346553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:51.377908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.455064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 833/1673] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:51.460494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:51.463214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:51.679483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b95187-997c-455a-a0e7-cc26cd9dc240 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:51.687514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:51.688126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.688442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.688716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] PATH_INFO: `/v3/projects/704e0d3c9b47411babb58e57ba7cb00b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.689948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=704e0d3c9b47411babb58e57ba7cb00b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:51.691941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:51.726972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1066a21c-32fb-478b-8d6b-4500d68d3481 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:51.743379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:51.744470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.744782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.745098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.790583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:51.803174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334bc302-f70a-4984-866b-eb8c3e8a69c5 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:51.836941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 834/1674] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:51.840740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:51.841614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:51.843722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:51.977181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:51.978543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:52.001400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '704e0d3c9b47411babb58e57ba7cb00b', 'request_id': 'req-ee1fb88c-b855-4977-ac4a-dc929e669ad0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.015333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '704e0d3c9b47411babb58e57ba7cb00b', 'request_id': 'req-ee1fb88c-b855-4977-ac4a-dc929e669ad0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.106018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '704e0d3c9b47411babb58e57ba7cb00b', 'request_id': 'req-ee1fb88c-b855-4977-ac4a-dc929e669ad0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.223551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 835/1675] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:52.225398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:52.249554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487d2ca6-7969-45ea-9631-019dca23d8bf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.262104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] The token cache is being invalidate because project 704e0d3c9b47411babb58e57ba7cb00b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:01:52.262554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.262823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.263472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.263759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.264037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.264788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ee1fb88c-b855-4977-ac4a-dc929e669ad0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:52.264991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:52.265400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.267301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.274102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 841/1676] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:01:51 2022] DELETE /identity/v3/projects/704e0d3c9b47411babb58e57ba7cb00b => generated 0 bytes in 928 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:52.281381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6333b-66fe-4a50-aa22-f304d0257e62 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.282651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6333b-66fe-4a50-aa22-f304d0257e62 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.282651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6333b-66fe-4a50-aa22-f304d0257e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.337584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 836/1677] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:52.337584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:52.337584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:52.369255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35d5881-06fe-4663-9cab-3ce37ade0e24 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.369376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7ec6333b-66fe-4a50-aa22-f304d0257e62 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:52.370576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7ec6333b-66fe-4a50-aa22-f304d0257e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:52.394585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 842/1678] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:52.405697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.412346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.412346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.413340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.414002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.415688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.416867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.421386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.427925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.481153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:52.486821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:52.508550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 843/1679] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:01:52.513879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:52.540227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6297ed3c-28a4-4526-b187-e436f796c599 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.550074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.550984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.551440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.553378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.553378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:01:52.553378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.557030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.604806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 844/1680] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:01:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:01:52.605924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:52.641857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b90ce8e-43f0-4115-b84e-c95840b7ffa0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.657048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.657819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.658168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.658509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.659242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.661459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.667417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:52.675158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 837/1681] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/projects => generated 385 bytes in 349 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:52.682708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:52.843167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823cfd8c-a267-4ea3-82dd-4a9e896e6815 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:52.878222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:52.879243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:52.879753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:52.881155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:52.882072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:52.893922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:52.917922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 845/1682] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/projects => generated 377 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:52.922394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.047692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 838/1683] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/users => generated 293 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:53.059508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.089637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69086017-b261-4a01-9a6b-76c3789818ca admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.257834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9955abb-0d10-4504-9037-68f83443503a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.283780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.284445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.290111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.291875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.292640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.303286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f27a80-357c-40ab-ad00-05702f084367 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.494470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 846/1684] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:52 2022] POST /identity/v3/users => generated 289 bytes in 570 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:01:53.494470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bad73-d69e-4a45-814e-79185117791a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.494470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bad73-d69e-4a45-814e-79185117791a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.494470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0bad73-d69e-4a45-814e-79185117791a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.503584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.504478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 847/1685] 10.222.0.97 () {54 vars in 912 bytes} [Thu Sep 8 14:01:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:01:53.504585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.505007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.505901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bad73-d69e-4a45-814e-79185117791a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.506462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.507248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.511196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14703a8e-4a74-4324-b2f9-09635ab6f000 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.549933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 839/1686] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 496 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:53.554499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0bad73-d69e-4a45-814e-79185117791a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.562866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.563263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.563551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.588227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.592571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.592907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.593259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.593905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.595570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.650214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 848/1687] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:53.651422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.652002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:53.658071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.683093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956277a2-c3bd-4733-9185-824bf055ae94 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.701475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.702365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.702854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.704079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.705445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.712505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffaedc-d4f3-4dfd-b044-79983dd7870c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.742063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 849/1688] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:53.743355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:53.746002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.749100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.749714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.843884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 840/1689] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:53.844950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.848967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] MFA Rules not processed for user `c18dc6b5898c4e1aae71a028b4840bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:53.851412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.876112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ef688b-82c7-41ec-ad37-3c46d576f209 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.887547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.888229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.890082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.890377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.892056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:53.897770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:53.925476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 841/1690] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:53.929747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.955730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cddb4-5c83-4e8f-ad72-78c5b27b4b29 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:53.967895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:53.968636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:53.968916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:53.969218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] PATH_INFO: `/v3/projects/ae844e83c79e43e1940e1458605b88d9/users/3571918eb0cf4659ae8e8129b10786d1/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:53.992885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 850/1691] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:53.994289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:53.998114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:53.999297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae844e83c79e43e1940e1458605b88d9, user_id=3571918eb0cf4659ae8e8129b10786d1, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.002560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c6b85a-d1e2-47bb-8ea4-9f21f4c93231 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.014572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92821b50-13e8-4743-a950-3fd21368a78d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.020644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.022116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.022941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.023469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.034482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.051433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.100872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 851/1692] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:54.102473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.133283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e303104e-14ad-4650-bf59-4d886b90bcf6 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.156298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.157117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.157279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.157547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] PATH_INFO: `/v3/projects/510c702dfdf6457aabafadfc3325ff2f/users/3c6313fe732b464ba776ffdd9499954d/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.187247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 842/1693] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:53 2022] PUT /identity/v3/projects/ae844e83c79e43e1940e1458605b88d9/users/3571918eb0cf4659ae8e8129b10786d1/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.195504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510c702dfdf6457aabafadfc3325ff2f, user_id=3c6313fe732b464ba776ffdd9499954d, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:54.198442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:54.198670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766da34-f5a8-4c05-aabd-95fffdfc0bef None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.198804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766da34-f5a8-4c05-aabd-95fffdfc0bef None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.199647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d766da34-f5a8-4c05-aabd-95fffdfc0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.328281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d766da34-f5a8-4c05-aabd-95fffdfc0bef None None] MFA Rules not processed for user `3571918eb0cf4659ae8e8129b10786d1`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.339528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d766da34-f5a8-4c05-aabd-95fffdfc0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.430091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 852/1694] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:54 2022] PUT /identity/v3/projects/510c702dfdf6457aabafadfc3325ff2f/users/3c6313fe732b464ba776ffdd9499954d/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:54.431274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:54.438536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.606317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 843/1695] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:54.613600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.615085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.615704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:54.735171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5800341e-8b99-46ab-b716-9da0b8e36f70 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:54.760287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:54.761363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:54.762088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] MFA Rules not processed for user `3c6313fe732b464ba776ffdd9499954d`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:54.764575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:54.766530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:54.766872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.003209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.016882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 844/1696] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 410 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:55.017420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.018144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.050015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 853/1697] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:55.052054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.053606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:55.152054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51571f10-bd92-4b97-824d-c15d6832b869 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.159239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.160017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.160997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.161530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.162459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.166389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.272104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564aefc1-0d96-4a78-8f89-5bf59b4defcf service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.351670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.351670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.353506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 845/1698] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/projects => generated 387 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:55.355973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.355973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] PATH_INFO: `/v3/users/6f87715ef916491e889c3b9f7322fc19` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.356423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:55.358238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f87715ef916491e889c3b9f7322fc19)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.373555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.567849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd33a030-1658-4e5f-a9a7-0aa49708ced4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:55.576589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:55.580462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:55.580974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:55.581392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:55.582259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:55.584665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:55.908580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 846/1699] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/projects => generated 375 bytes in 554 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:55.922091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:56.081265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f87715ef916491e889c3b9f7322fc19'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:56.081265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f87715ef916491e889c3b9f7322fc19'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:56.105277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f87715ef916491e889c3b9f7322fc19'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:56.107329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f87715ef916491e889c3b9f7322fc19'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:56.161532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad1af1b-2e44-4efd-9027-4beb17649bfa admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:56.164456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:56.182621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:56.183264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:56.184115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:56.185055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:56.185510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f87715ef916491e889c3b9f7322fc19'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:56.185741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:56.188067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a37bc2-3f1e-4d8d-a46a-f1845be49ad4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:56.516609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 854/1700] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:55 2022] DELETE /identity/v3/users/6f87715ef916491e889c3b9f7322fc19 => generated 0 bytes in 1462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:56.518058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 847/1701] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:55 2022] POST /identity/v3/users => generated 300 bytes in 609 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:56.520071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:56.522483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:56.522996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:56.526583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:56.527030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:56.675610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] MFA Rules not processed for user `9b0569d9de724d5fbe3f41cbc0687d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:56.678147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:56.702072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40494ee3-0787-48a6-a1a4-2b7a597f1c1c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:56.720045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:56.721819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:56.724799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:56.727972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] PATH_INFO: `/v3/users/ce143361f42046d29e6b191037331fbc` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:56.732872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce143361f42046d29e6b191037331fbc)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:56.742732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:56.917238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 848/1702] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:56.919088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:56.927854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:56.966146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e584276-594f-48ea-b2fc-b6846dad79df None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:56.982735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:56.993950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:56.994798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:56.994798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:56.995344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:57.002812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2853bee-0f4e-49bc-8e4f-20b83c467f9c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:57.324908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce143361f42046d29e6b191037331fbc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:57.324908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce143361f42046d29e6b191037331fbc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:57.335784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce143361f42046d29e6b191037331fbc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:57.337185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce143361f42046d29e6b191037331fbc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:57.395541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 849/1703] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:01:56 2022] POST /identity/v3/users => generated 294 bytes in 477 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:01:57.396945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:57.397231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:57.400635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:57.427163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce143361f42046d29e6b191037331fbc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:01:57.483611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:57.487892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:57.507733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 850/1704] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:57.509163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:57.513019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:57.539062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5d743f-c72a-4ad7-bb56-81e4273c2a97 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:57.547578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:57.548792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:57.549276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:57.549940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:57.551311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:57.555095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:57.609245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 851/1705] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:57.612934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:57.619922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:57.655815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 855/1706] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:01:56 2022] DELETE /identity/v3/users/ce143361f42046d29e6b191037331fbc => generated 0 bytes in 1138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:57.659063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:57.663196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:57.847222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883aa531-fa8c-41bd-8dd1-c10b93a5676f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:57.867084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:57.868071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:57.869283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:57.869793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:57.871149 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:57.873136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:57.905767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 856/1707] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:57.906971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:57.909849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:57.920726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b98e20-e457-4633-a375-2545bc7bd132 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:58.034404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52961771-f4b3-4982-bf8f-ba923aad015e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:58.044914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:58.048118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.048774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.049321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.075771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:58.081096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:58.134962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 857/1708] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:58.138066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:58.158830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:58.163693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.164311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.164800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.174347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:58.193752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f1402a-d1c1-44cd-b374-0c367c368cb7 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:58.209886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:58.212422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.213519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.214037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] PATH_INFO: `/v3/projects/f9466d70d4cb43a3969278e4b724440a/users/2137feec9a434b2fa45aeada5976664e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.263628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9466d70d4cb43a3969278e4b724440a, user_id=2137feec9a434b2fa45aeada5976664e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:58.266963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:58.371691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:58.375181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3192b915-9dc4-454a-a4cb-abeaff1a847f service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:58.405298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 852/1709] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 793 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:58.407697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.408572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.408572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.458323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 858/1710] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:58 2022] PUT /identity/v3/projects/f9466d70d4cb43a3969278e4b724440a/users/2137feec9a434b2fa45aeada5976664e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:58.461775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:58.494320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:58.497120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:58.613945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dbfcb9-5e18-43ed-8145-bb3aa011f76d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:58.627374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:58.629216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.630017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.630919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] PATH_INFO: `/v3/projects/d7ed1848eff24316acf6d6bba3f3347b/users/24b4dd5c94464efdbbb71197324d9153/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.665054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 853/1711] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:58.666877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:58.669841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ed1848eff24316acf6d6bba3f3347b, user_id=24b4dd5c94464efdbbb71197324d9153, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:58.703913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac72119c-df9e-48ab-8d70-997dd51f1a76 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:58.706748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:58.721175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:58.722698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:58.725233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:58.725691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:58.726566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:58.728624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:58.744815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 854/1712] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:58.746381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:58.767842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:58.864374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 859/1713] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:58 2022] PUT /identity/v3/projects/d7ed1848eff24316acf6d6bba3f3347b/users/24b4dd5c94464efdbbb71197324d9153/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:58.877143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:59.033326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f49c90-a622-4cdb-b6da-64efa9fc774d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:59.128551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130532a8-e2e8-4f35-83ea-204ce8486e96 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:59.139715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:59.139715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.140544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.141040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] PATH_INFO: `/v3/projects/f9466d70d4cb43a3969278e4b724440a/users/2137feec9a434b2fa45aeada5976664e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.147785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9466d70d4cb43a3969278e4b724440a, user_id=2137feec9a434b2fa45aeada5976664e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:59.154359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:59.188624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 860/1714] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:58 2022] PUT /identity/v3/projects/f9466d70d4cb43a3969278e4b724440a/users/2137feec9a434b2fa45aeada5976664e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:59.190779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:59.250834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:59.251502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.251824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.252285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.284109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:59.285776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627c8d78-fad4-4c87-b010-cc27eb8171d1 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:59.317211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 855/1715] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 570 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:01:59.320491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.320718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.322823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.421615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] MFA Rules not processed for user `2137feec9a434b2fa45aeada5976664e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:59.425370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:59.451457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc114e7-d790-4240-96c4-97d0ec217fc7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:59.479815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:59.481453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.481453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.481755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.482297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:59.486375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:59.542452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 861/1716] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:01:59.549049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:59.596486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7dbb5-6a32-4d02-b8ec-a69caa49c66e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:01:59.619990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:01:59.619990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.619990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.619990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] PATH_INFO: `/v3/projects/d7ed1848eff24316acf6d6bba3f3347b/users/24b4dd5c94464efdbbb71197324d9153/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.622395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7ed1848eff24316acf6d6bba3f3347b, user_id=24b4dd5c94464efdbbb71197324d9153, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:01:59.631527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fd2f57-91a8-451b-ba1e-fad715364e10 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:01:59.666123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 862/1717] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:01:59 2022] PUT /identity/v3/projects/d7ed1848eff24316acf6d6bba3f3347b/users/24b4dd5c94464efdbbb71197324d9153/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:01:59.678428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfdd8f-2cfe-4dc6-ada3-c7f2c97598fb None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:01:59.678813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfdd8f-2cfe-4dc6-ada3-c7f2c97598fb None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:01:59.679099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abfdd8f-2cfe-4dc6-ada3-c7f2c97598fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:01:59.707501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 856/1718] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 391 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:01:59.718782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:01:59.721234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:59.750498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:01:59.763544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0abfdd8f-2cfe-4dc6-ada3-c7f2c97598fb None None] MFA Rules not processed for user `24b4dd5c94464efdbbb71197324d9153`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:01:59.765921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0abfdd8f-2cfe-4dc6-ada3-c7f2c97598fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:01:59.910814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69224a8d-5a10-47d4-80b5-2db22a9feacb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:00.006049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 863/1719] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:01:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:00.006049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:00.006049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:00.006049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:00.032555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:00.033491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:00.033962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:00.034409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:00.126202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:00.127515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:00.194528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:00.196001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:00.239581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 857/1720] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:00.243555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:00.250404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:00.301278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740dd952-1704-4f1c-8ae2-88c18a21db67 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:00.309894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:00.310605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:00.311166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:00.311309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:00.323632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:00.392178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 864/1721] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:02:00 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:00.403560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:00.410613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:00.516905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:00.518368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:00.547250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 858/1722] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:00.548834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:00.612468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85094c2e-8ffa-4048-85eb-e029b5f26ee0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:00.623980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:00.625032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:00.625403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:00.625753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] PATH_INFO: `/v3/projects/769f033e409344a18a3e74c7b9314caa` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:00.630073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=769f033e409344a18a3e74c7b9314caa)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:00.632255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:00.782487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b40fb0-ee0b-4629-985d-671f95cc68c7 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:00.809756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:00.812348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:00.813067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:00.815726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] PATH_INFO: `/v3/users/3571918eb0cf4659ae8e8129b10786d1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:00.825147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3571918eb0cf4659ae8e8129b10786d1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:00.831531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:00.988637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '769f033e409344a18a3e74c7b9314caa', 'request_id': 'req-410aa2b1-1b2d-44f4-8a25-03469ac27c52'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.003910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '769f033e409344a18a3e74c7b9314caa', 'request_id': 'req-410aa2b1-1b2d-44f4-8a25-03469ac27c52'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.013452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '769f033e409344a18a3e74c7b9314caa', 'request_id': 'req-410aa2b1-1b2d-44f4-8a25-03469ac27c52'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.300040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] The token cache is being invalidate because project 769f033e409344a18a3e74c7b9314caa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:01.301255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.304069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.313038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 865/1723] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:00 2022] DELETE /identity/v3/projects/769f033e409344a18a3e74c7b9314caa => generated 0 bytes in 911 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:01.314443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:01.324910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:01.483070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3571918eb0cf4659ae8e8129b10786d1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.483070 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3571918eb0cf4659ae8e8129b10786d1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.494328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3571918eb0cf4659ae8e8129b10786d1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.537020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3571918eb0cf4659ae8e8129b10786d1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.538615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3571918eb0cf4659ae8e8129b10786d1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:01.790161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410aa2b1-1b2d-44f4-8a25-03469ac27c52 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:01.860322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:01.860814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 859/1724] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:00 2022] DELETE /identity/v3/users/3571918eb0cf4659ae8e8129b10786d1 => generated 0 bytes in 1313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:01.863588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:01.863970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:01.866212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:01.866573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:01.867152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:01.874198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:01.988750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:01.990608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb586040-bebc-4b93-8661-705e50fad186 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:02.036244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 866/1725] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:02:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 722 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:02:02.048716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:02.048716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:02.048716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:02.161004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:02.179586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:02.190277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d1d0c-d0fc-46ea-83ec-ba8455d24477 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:02.207149 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:02.208423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:02.209144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:02.209634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] PATH_INFO: `/v3/users/74f0118986d94e1d8c829effd35cf3a7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:02.212194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] RBAC: Authorizing `identity:delete_user(user_id=74f0118986d94e1d8c829effd35cf3a7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:02.236794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 867/1726] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:02.237138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:02.517080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:02.518876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:02.703512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:02.854247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74f0118986d94e1d8c829effd35cf3a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.860678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74f0118986d94e1d8c829effd35cf3a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:02.938906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74f0118986d94e1d8c829effd35cf3a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.005309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6088c7cc-7043-49fd-b795-733a22cce53b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.014096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74f0118986d94e1d8c829effd35cf3a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.017855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74f0118986d94e1d8c829effd35cf3a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.022390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.023075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.023365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.023642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.029481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.065336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.066854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970bf3ab-2480-4585-bddd-e71a61a487e6 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:03.098271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 868/1727] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:03.099911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.100462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.100920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.160594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:03.165840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.191133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 860/1728] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:01 2022] DELETE /identity/v3/users/74f0118986d94e1d8c829effd35cf3a7 => generated 0 bytes in 1330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:03.209154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 869/1729] 10.222.0.77 () {58 vars in 1015 bytes} [Thu Sep 8 14:02:03 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:03.291571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:03.294048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.398084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:03.401682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:03.419823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fc1dba-17b6-416d-b8ed-2a168d067f46 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.542327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051a5c84-1d7d-4890-9d5b-a26ba4d456af admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:03.550769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.551783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.552425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:03.553428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:03.553879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.554108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:03.554479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.555018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] PATH_INFO: `/v3/projects/8dd4951d620c4bf4b5b0ba9f257424e3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:03.557214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8dd4951d620c4bf4b5b0ba9f257424e3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.561046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:03.574575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:03.576737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb084d17-d02c-43f8-85f1-5ae6a6e800f1 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:03.588028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 870/1730] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:03 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:03.849702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8dd4951d620c4bf4b5b0ba9f257424e3', 'request_id': 'req-b6c69be4-0e36-4805-b2db-0b8bfa436361'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.869056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8dd4951d620c4bf4b5b0ba9f257424e3', 'request_id': 'req-b6c69be4-0e36-4805-b2db-0b8bfa436361'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:03.894395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8dd4951d620c4bf4b5b0ba9f257424e3', 'request_id': 'req-b6c69be4-0e36-4805-b2db-0b8bfa436361'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:04.371320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] The token cache is being invalidate because project 8dd4951d620c4bf4b5b0ba9f257424e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:04.371320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:04.371320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b6c69be4-0e36-4805-b2db-0b8bfa436361 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:04.371320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 861/1731] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:03 2022] DELETE /identity/v3/projects/8dd4951d620c4bf4b5b0ba9f257424e3 => generated 0 bytes in 901 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:04.375249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.375249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.375249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.469373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:04.469373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:04.723618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 871/1732] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:04.737329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.738116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.741679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.871779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:04.880548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:04.929431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 862/1733] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:04.930487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:04.953668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411e9aee-ba83-4900-965e-d9f4a569a8e8 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:04.966575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:04.967172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:04.967507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:04.967937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:04.968582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:04.968913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:04.975506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f90bce4-963e-467b-abe4-152e69a3a078 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:05.018253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 872/1734] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:05.019632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:05.035184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5ac1c7-0932-48e4-84f7-6b6ec504e1a9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:05.053916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:05.054937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:05.054937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:05.055248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:05.056173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:05.056398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:05.056568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:05.058946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:05.062881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:05.180309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] MFA Rules not processed for user `c18dc6b5898c4e1aae71a028b4840bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:05.193760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:05.249063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 863/1735] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:05 2022] POST /identity/v3/projects => generated 363 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:05.253786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:05.257953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:05.277780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:05.418399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 873/1736] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:05.449703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:05.453461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904ccee-2c3e-451a-9c15-7ab1578481c4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:05.539173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef9bf90-b2b4-45ad-a982-f09319decd32 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:05.585221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:05.592434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:05.592434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:05.592434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] PATH_INFO: `/v3/projects/ae844e83c79e43e1940e1458605b88d9` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:05.597997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae844e83c79e43e1940e1458605b88d9)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:05.601340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:05.765486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:05.766415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:05.766415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:05.766415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:05.769465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:05.769749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:05.975212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 874/1737] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:05 2022] POST /identity/v3/users => generated 282 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:05.976576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:06.009454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae844e83c79e43e1940e1458605b88d9', 'request_id': 'req-97f1dd27-394f-485c-9575-faed1a0b3895'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.012521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92edcd9-2362-403c-b5a8-0ea775fa4574 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:06.040761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae844e83c79e43e1940e1458605b88d9', 'request_id': 'req-97f1dd27-394f-485c-9575-faed1a0b3895'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.053916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae844e83c79e43e1940e1458605b88d9', 'request_id': 'req-97f1dd27-394f-485c-9575-faed1a0b3895'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.207329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:06.211488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:06.211863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:06.212174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:06.212964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:06.219207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:06.220536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] The token cache is being invalidate because project ae844e83c79e43e1940e1458605b88d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:06.221107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.222647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.224932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 864/1738] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:05 2022] DELETE /identity/v3/projects/ae844e83c79e43e1940e1458605b88d9 => generated 0 bytes in 972 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:06.232524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:06.235692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:06.304404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f1dd27-394f-485c-9575-faed1a0b3895 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:06.312614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 875/1739] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:05 2022] GET /identity/v3/roles => generated 1420 bytes in 331 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:06.318166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:06.329193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:06.339220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:06.343804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:06.344517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:06.346129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] PATH_INFO: `/v3/users/2137feec9a434b2fa45aeada5976664e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:06.351128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2137feec9a434b2fa45aeada5976664e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:06.357004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:06.368263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:06.379119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fc7dd4-afaa-4d36-b705-e5e901a96ad7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:06.391627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:06.392650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:06.392979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:06.393309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:06.394024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:06.395958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:06.450296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 876/1740] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:06.454479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:06.491844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e138cd-846c-4a45-a0d2-272bf24b7080 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:06.505127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:06.505888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:06.506203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:06.506642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] PATH_INFO: `/v3/projects/3c471e6889aa4a1ea50338db56671e98/users/1bc63e5978d44202a277e6261d93d65d/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:06.546928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c471e6889aa4a1ea50338db56671e98, user_id=1bc63e5978d44202a277e6261d93d65d, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:06.550089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8537dc-bc18-4c1b-a780-09c32657c675 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:06.733557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2137feec9a434b2fa45aeada5976664e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.735614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2137feec9a434b2fa45aeada5976664e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.755204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2137feec9a434b2fa45aeada5976664e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.794997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2137feec9a434b2fa45aeada5976664e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.795865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2137feec9a434b2fa45aeada5976664e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:06.879699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 877/1741] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:06 2022] PUT /identity/v3/projects/3c471e6889aa4a1ea50338db56671e98/users/1bc63e5978d44202a277e6261d93d65d/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:06.894172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:06.894731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:06.895396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:06.926607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 865/1742] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:06 2022] DELETE /identity/v3/users/2137feec9a434b2fa45aeada5976664e => generated 0 bytes in 697 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:06.992031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] MFA Rules not processed for user `1bc63e5978d44202a277e6261d93d65d`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:06.997547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:07.240173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 878/1743] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:06 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:07.240348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:07.241996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:07.528202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458df888-e455-4352-a0cd-aeabd0e0d752 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:07.540725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:07.544883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:07.546160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:07.548307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:07.549622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:07.561367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:07.627616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:07.630838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:07.665044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bcec00-8a1b-459f-8c4b-ed17eb009a28 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:07.680779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:07.681461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:07.681985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:07.682446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] PATH_INFO: `/v3/projects/493f7b105deb418db42b396ffe7454c3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:07.684843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] RBAC: Authorizing `identity:delete_project(project_id=493f7b105deb418db42b396ffe7454c3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:07.689802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:07.899944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 866/1744] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:07 2022] POST /identity/v3/projects => generated 363 bytes in 662 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:07.908027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:08.061154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3161d60-2db3-4bb1-9241-5a9ec24d9323 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:08.076971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:08.078903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.080877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.081790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.083320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:08.097301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:08.151109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '493f7b105deb418db42b396ffe7454c3', 'request_id': 'req-f8ba2878-0c9b-400d-ac66-67eaf24b747e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:08.186664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '493f7b105deb418db42b396ffe7454c3', 'request_id': 'req-f8ba2878-0c9b-400d-ac66-67eaf24b747e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:08.224694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '493f7b105deb418db42b396ffe7454c3', 'request_id': 'req-f8ba2878-0c9b-400d-ac66-67eaf24b747e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:08.318134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 867/1745] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:07 2022] POST /identity/v3/users => generated 288 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:08.327386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:08.350481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e743542f-ab19-46b5-99c1-1f38e002d94b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:08.373525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] The token cache is being invalidate because project 493f7b105deb418db42b396ffe7454c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:08.374479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:08.375668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f8ba2878-0c9b-400d-ac66-67eaf24b747e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:08.385553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 879/1746] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:07 2022] DELETE /identity/v3/projects/493f7b105deb418db42b396ffe7454c3 => generated 0 bytes in 753 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:08.387502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57366-e991-4f09-941c-56cd444d3264 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.402276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57366-e991-4f09-941c-56cd444d3264 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.402276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d57366-e991-4f09-941c-56cd444d3264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.507914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e0d57366-e991-4f09-941c-56cd444d3264 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:08.521378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e0d57366-e991-4f09-941c-56cd444d3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:08.552712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:08.553386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.553668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.553942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.554605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:08.557265 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:08.615001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 868/1747] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:08.627599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:08.635755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:08.665257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031e282e-f967-4532-942f-1961583ea49a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:08.676917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:08.678004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.678529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.679883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] PATH_INFO: `/v3/projects/300f3c5326b84a1bae47eb9f595f6063/users/e8560b3031ec4a639f2903a64ce9a0ff/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.704536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=300f3c5326b84a1bae47eb9f595f6063, user_id=e8560b3031ec4a639f2903a64ce9a0ff, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:08.708424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:08.728626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 880/1748] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 342 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:08.731275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65383e-b128-49d0-805d-90ed278175df None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.732962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65383e-b128-49d0-805d-90ed278175df None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.736655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65383e-b128-49d0-805d-90ed278175df None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.744412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 881/1749] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:02:08 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:02:08.749261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2667d1-76ae-465b-978a-e4d26ea5a0c3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.749839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2667d1-76ae-465b-978a-e4d26ea5a0c3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.750142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2667d1-76ae-465b-978a-e4d26ea5a0c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.825611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0a2667d1-76ae-465b-978a-e4d26ea5a0c3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:08.828551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0a2667d1-76ae-465b-978a-e4d26ea5a0c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:08.847851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 882/1750] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:08.856931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:08.856931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:08.856931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:08.894987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:08.910079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:08.912704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 869/1751] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:08 2022] PUT /identity/v3/projects/300f3c5326b84a1bae47eb9f595f6063/users/e8560b3031ec4a639f2903a64ce9a0ff/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:08.915452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:08.943222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:09.045916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ead00-c507-4890-b2fd-4cbf0509e3af admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.052778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.053457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.053743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.055410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.056176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:09.060952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.061301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.081940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 870/1752] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:09.083090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.117798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f05156-b365-4e88-bb02-a423196a3434 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.127857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.129286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.129996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.130698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.132056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.137345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.154883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 883/1753] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:02:08 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:09.154883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.168554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acae1c9a-0d83-47bf-b9d9-e6b686b1b3e6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.175499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 871/1754] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:09.179474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.180564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.181119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.181564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.182422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.185378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.190017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.226037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f5883e-2970-4626-a6b4-bb5f5e34e043 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.250129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.252113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.252701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.253196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] PATH_INFO: `/v3/projects/300f3c5326b84a1bae47eb9f595f6063/users/e8560b3031ec4a639f2903a64ce9a0ff/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.256330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=300f3c5326b84a1bae47eb9f595f6063, user_id=e8560b3031ec4a639f2903a64ce9a0ff, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.261158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ff70b-06dc-480b-80f4-0dd6157473c8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.278899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 872/1755] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:09 2022] PUT /identity/v3/projects/300f3c5326b84a1bae47eb9f595f6063/users/e8560b3031ec4a639f2903a64ce9a0ff/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:09.330868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.331531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.331859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.397590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 884/1756] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:09 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:09.399008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.401367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:09.405181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] MFA Rules not processed for user `e8560b3031ec4a639f2903a64ce9a0ff`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:09.406571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:09.541397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97159ca-2ccd-49bc-bada-b08b46882d8d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.548494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 873/1757] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:09 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:09.548745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.549172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.553763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.554312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.555101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.582777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.589245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.642116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 885/1758] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:09.657577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.659086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:09.716202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17726ebc-0f13-400e-a23f-9ac908a033c5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.723435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.724074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.724395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.724679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.725596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.728577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.802253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae002-e5e6-4b66-a64e-21af3d828dfc service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:09.818743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:09.823175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.824932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.837605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.843531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:09.845235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b2ec4-5b90-4951-81d4-aebd8c7d31d6 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:09.869854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 886/1759] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:09 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:09.872289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:09.872577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:09.872848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:09.915942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] MFA Rules not processed for user `9b0569d9de724d5fbe3f41cbc0687d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:09.918264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:09.974789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 874/1760] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:09 2022] POST /identity/v3/users => generated 289 bytes in 426 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:09.979390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:09.985834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:10.018732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f67091-0818-45b8-a40f-f575d2337ade admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.031003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.031758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.032049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.034688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] PATH_INFO: `/v3/projects/f9466d70d4cb43a3969278e4b724440a` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.034688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9466d70d4cb43a3969278e4b724440a)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:10.036229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:10.043148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 887/1761] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:09 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:10.044461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:10.061946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0ea52c-c546-4fba-977a-3f541bf215b1 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.069424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.072989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.072989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.072989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.072989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:10.085159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:10.101977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 888/1762] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:10.111167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:10.138597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478bee72-a0fa-471b-a5f4-9d9e6a195b76 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.150655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.154472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.154472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.154472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.154472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:10.156989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:10.185204 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 889/1763] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:10.190195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:10.227660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef4f787-4d28-4965-aff6-28cc53702d0d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.236774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.238110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.238699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.239216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] PATH_INFO: `/v3/projects/b7c7a6ad527c48ecb999a1338f92d87e/users/006d2fa12c7e4851a852475ce648d82d/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.274407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7c7a6ad527c48ecb999a1338f92d87e, user_id=006d2fa12c7e4851a852475ce648d82d, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:10.278207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:10.415552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 890/1764] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:10 2022] PUT /identity/v3/projects/b7c7a6ad527c48ecb999a1338f92d87e/users/006d2fa12c7e4851a852475ce648d82d/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:10.416521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:10.441114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9466d70d4cb43a3969278e4b724440a', 'request_id': 'req-bb662784-ce39-48b4-8277-a718afb333ed'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:10.460874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9466d70d4cb43a3969278e4b724440a', 'request_id': 'req-bb662784-ce39-48b4-8277-a718afb333ed'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:10.489878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9466d70d4cb43a3969278e4b724440a', 'request_id': 'req-bb662784-ce39-48b4-8277-a718afb333ed'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:10.543350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d6045d-4a87-4b7a-9cdf-389b6f121ca1 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.638943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] The token cache is being invalidate because project f9466d70d4cb43a3969278e4b724440a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:10.647269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:10.647269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb662784-ce39-48b4-8277-a718afb333ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:10.647269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 875/1765] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:09 2022] DELETE /identity/v3/projects/f9466d70d4cb43a3969278e4b724440a => generated 0 bytes in 664 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:10.647269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b207e-8f6e-4778-94e1-7be51dcfa8d1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.647857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b207e-8f6e-4778-94e1-7be51dcfa8d1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.648569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b207e-8f6e-4778-94e1-7be51dcfa8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.650609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.651311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.651601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.651882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.661074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:10.755043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:10.763373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-262b207e-8f6e-4778-94e1-7be51dcfa8d1 None None] MFA Rules not processed for user `006d2fa12c7e4851a852475ce648d82d`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:10.767390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-262b207e-8f6e-4778-94e1-7be51dcfa8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:10.885970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:10.885970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:10.911112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 891/1766] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:10.911391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:10.920693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:10.949108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c861b3f-53a3-4068-a7a7-3b4f85cdcd89 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:10.958525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:10.959801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:10.960424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:10.960990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:10.989630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:11.026065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 876/1767] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:10 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:11.027822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.028395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.028885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.052611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:11.054360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:11.130905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 877/1768] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:11.132067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.147342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:11.163514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1c9f50-f070-479e-820d-1d313c5aa3bb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.169629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.170948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.171569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.172296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.180983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.180983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.268359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.270166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f71e6a-193e-4be9-b5a3-1db1890b4527 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.290146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 892/1769] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:11.293905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.293905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.296515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.353850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 878/1770] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:11 2022] POST /identity/v3/projects => generated 377 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:11.357858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.385372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:11.396214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:11.486630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374b9d5c-d5f8-4897-8372-498229338903 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.492223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.492913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.493550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.493892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.494702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.497488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.639023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 879/1771] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:11 2022] POST /identity/v3/users => generated 289 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:11.642151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.661933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 893/1772] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:11.665631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.669217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9776370-5115-47ce-80b4-fbe5aca8f096 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.677723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c89f0-57b5-4279-a507-3a3e9b674ab8 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.691180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.693981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.693981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.693981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.693981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:11.695759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.695979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.696690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.696996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.699485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.699939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.701472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.705309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.737378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 894/1773] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:11.744079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.750576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 880/1774] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:11.765402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.784756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a7c620-2967-4e54-a11a-7f50407a64bf admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.803176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ce729a-3387-48ee-815d-8cb7199d0541 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.812003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.814302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.815455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.815959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.816130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.816459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.817323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.817642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:11.818386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.819512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:11.820399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.825972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:11.880664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 881/1775] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:11 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:11.892411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:11.966104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afb991b-53ab-4d3f-b925-be31e471173f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:11.994861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:11.995652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:11.996577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:11.996932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] PATH_INFO: `/v3/projects/20a2a3659dab4a83852fa9442758f4b8/users/d808a4e7226b41789677b8595513798f/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:12.030123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20a2a3659dab4a83852fa9442758f4b8, user_id=d808a4e7226b41789677b8595513798f, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:12.036398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be52d901-12f6-4866-8ace-395bf49812c0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:12.073804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 895/1776] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:11 2022] POST /identity/v3/projects => generated 399 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:12.083422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:12.210072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 882/1777] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:11 2022] PUT /identity/v3/projects/20a2a3659dab4a83852fa9442758f4b8/users/d808a4e7226b41789677b8595513798f/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:12.212043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862eef-179d-4850-8744-525d14b955af None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:12.212344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862eef-179d-4850-8744-525d14b955af None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:12.212857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f862eef-179d-4850-8744-525d14b955af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:12.309509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5f862eef-179d-4850-8744-525d14b955af None None] MFA Rules not processed for user `d808a4e7226b41789677b8595513798f`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:12.310736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5f862eef-179d-4850-8744-525d14b955af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:12.326152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb2cf9-0e5d-44d9-b8d9-3a40329206dc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:12.571207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:12.572510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 883/1778] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 361 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 08 14:02:12.575262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f862eef-179d-4850-8744-525d14b955af None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:12.617553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:12.620686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:12.622993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:12.625145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:12.627037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:12.637556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:12.803757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f862eef-179d-4850-8744-525d14b955af None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:12.811801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:12.812807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:12.813205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:12.813416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:12.814175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:12.816652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:12.949293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 896/1779] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:12 2022] POST /identity/v3/users => generated 300 bytes in 867 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:12.954147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:12.975834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a741635c-5c48-4d2f-be68-e628c6da200b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.004342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.016894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.016894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.016894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.018190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.024204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 884/1780] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:12 2022] POST /identity/v3/projects => generated 379 bytes in 450 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:13.025240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.028825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.068355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 897/1781] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:12 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:13.069981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.109696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92f05f-386f-4f44-94ea-b1457e9e793a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.185384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bffa69b-4fc0-4330-ac05-ba1bd054ecad admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.227849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.231307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.234698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.235432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.242542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.248443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.310500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.313107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.319172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.320490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.325369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.330739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.364595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 898/1782] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:13.365737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.378646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffaaf1b-ffac-4995-aa2e-3490e806fca7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.384640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.385469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.385837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.386145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] PATH_INFO: `/v3/projects/785e883df3fe4eda83165b09544c68b0/users/d985abff33e4413c87709f1d83c0fd0b/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.393884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 885/1783] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:13 2022] POST /identity/v3/users => generated 296 bytes in 366 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:13.409562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.428068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faded4a3-57be-4959-88df-b3886f86c8c5 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.442178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.442961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.443499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.443867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.444889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.448110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.451934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785e883df3fe4eda83165b09544c68b0, user_id=d985abff33e4413c87709f1d83c0fd0b, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.457752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821ab59-2db2-4410-bb26-cb7b90e2e092 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.505361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 886/1784] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:13.512102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.536367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc01f82-30a1-4109-ada7-6e14fb4e32bf admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.546696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.547598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.548230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.548772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] PATH_INFO: `/v3/projects/70c4454491554247a58be4b95f5c693c/users/e1dd5839a2b6495e90938ef6852a8d5b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.572553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c4454491554247a58be4b95f5c693c, user_id=e1dd5839a2b6495e90938ef6852a8d5b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.582606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:13.685843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 899/1785] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:13 2022] PUT /identity/v3/projects/785e883df3fe4eda83165b09544c68b0/users/d985abff33e4413c87709f1d83c0fd0b/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:13.699772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.700133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.700515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.734164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] MFA Rules not processed for user `d985abff33e4413c87709f1d83c0fd0b`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:13.735678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:13.841751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 887/1786] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:13 2022] PUT /identity/v3/projects/70c4454491554247a58be4b95f5c693c/users/e1dd5839a2b6495e90938ef6852a8d5b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:13.846525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.883309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 900/1787] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:13 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:13.884635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:13.966742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7e08c-78e8-4579-899e-efe784da8bdd admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:13.975894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:13.976925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:13.977346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:13.977792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:13.979458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:13.985519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.007243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 888/1788] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:14.008113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.021892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d35a2-b090-4614-b22d-abc2bebaabee admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.034112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.036434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.037166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.037439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] PATH_INFO: `/v3/projects/70c4454491554247a58be4b95f5c693c/users/e1dd5839a2b6495e90938ef6852a8d5b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.044828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] RBAC: Authorizing `identity:create_grant(project_id=70c4454491554247a58be4b95f5c693c, user_id=e1dd5839a2b6495e90938ef6852a8d5b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.045046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5fcad8-9ed9-4169-80fd-f034dc0a7095 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.051884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8576da-ed1b-49ab-8033-8aa102ba7209 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.063003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.063637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.063919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.064268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.065116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.072873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.079051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 889/1789] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:14 2022] PUT /identity/v3/projects/70c4454491554247a58be4b95f5c693c/users/e1dd5839a2b6495e90938ef6852a8d5b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:14.089748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.090275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.090687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.128074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] MFA Rules not processed for user `e1dd5839a2b6495e90938ef6852a8d5b`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:14.129620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:14.220282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 890/1790] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:14 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:14.240348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.297290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 901/1791] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:13 2022] POST /identity/v3/projects => generated 401 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:14.298531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.399599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa565d0b-8d62-4548-8e1a-52811f1a92c2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.436999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6307a78b-5870-462c-b5d7-5f546073c2bd admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.458571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.459563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.459810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.461621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.462385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.464509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.503946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.504722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.505081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.505433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.580878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.582520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.592767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 891/1792] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:14.617787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 902/1793] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:14 2022] POST /identity/v3/users => generated 307 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:14.620537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.633583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d5b764-8c99-41c5-a21e-73d5d6514b69 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.638915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.639566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.639846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.640121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.640808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.642508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.653603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 892/1794] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:14.664512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.675733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e99f2e3-26f3-41e9-b989-1b6f6fc846d9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.680820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.681524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.681805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.682076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] PATH_INFO: `/v3/projects/e48d95dff0b548f984e70cb884027967/users/b625b6c0e0894d208fa14b564c7e20d2/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.700025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48d95dff0b548f984e70cb884027967, user_id=b625b6c0e0894d208fa14b564c7e20d2, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.704321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.789396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.791494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:14.809252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c6b354-0e0d-4877-a1df-9a069b988813 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:14.813665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:14.814352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:14.814639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:14.814922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:14.882543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 903/1795] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:14 2022] PUT /identity/v3/projects/e48d95dff0b548f984e70cb884027967/users/b625b6c0e0894d208fa14b564c7e20d2/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:14.887496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:14.910132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:14.911723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:14.923550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 893/1796] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:14.996219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b5da43-1cba-4ccc-8045-5b8be3c58636 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:15.006921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:15.007987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:15.008616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:15.009226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:15.010433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:15.013901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709ac91c-a47f-4169-886b-f89d3dd11450 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:15.037522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 904/1797] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:15.038946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:15.052064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b711724-ab41-4175-b185-3fb50498e310 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:15.056759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:15.057431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:15.057715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:15.057993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] PATH_INFO: `/v3/projects/e48d95dff0b548f984e70cb884027967/users/b625b6c0e0894d208fa14b564c7e20d2/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:15.060453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e48d95dff0b548f984e70cb884027967, user_id=b625b6c0e0894d208fa14b564c7e20d2, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:15.063802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:15.075539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 894/1798] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:15 2022] PUT /identity/v3/projects/e48d95dff0b548f984e70cb884027967/users/b625b6c0e0894d208fa14b564c7e20d2/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:15.088477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:15.088885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:15.089021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:15.123516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] MFA Rules not processed for user `b625b6c0e0894d208fa14b564c7e20d2`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:15.124859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:15.222302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 905/1799] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:15 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:15.238776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:15.347016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078585f-d722-4433-b8bf-20c82c319e91 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:15.355843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:15.356835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:15.357112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:15.357387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:15.370251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:15.372562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:15.384374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 895/1800] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:15 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:15.897045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:15.929687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:15.942628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92780d52-9658-49da-88c7-70e856a84e01 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:15.946811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:15.947511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:15.948001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:15.948413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:16.035256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:16.038054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2bcdb3-20d7-48c5-bdbe-e0e0fa33ac81 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:16.053804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 906/1801] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:15 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:16.446596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:18.713194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:18.715586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:18.908792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838f3323-6a3f-419f-b93b-c419a06d64a8 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:18.914321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:18.915077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:18.915363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:18.915667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] PATH_INFO: `/v3/users/1bc63e5978d44202a277e6261d93d65d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:18.917121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bc63e5978d44202a277e6261d93d65d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:18.919586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:19.265090 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bc63e5978d44202a277e6261d93d65d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.274525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bc63e5978d44202a277e6261d93d65d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.286976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bc63e5978d44202a277e6261d93d65d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.370565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bc63e5978d44202a277e6261d93d65d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.372061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bc63e5978d44202a277e6261d93d65d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:19.568746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 896/1802] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:18 2022] DELETE /identity/v3/users/1bc63e5978d44202a277e6261d93d65d => generated 0 bytes in 853 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:19.571981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:19.572819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:19.572959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:19.630749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:19.633186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:19.836104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 907/1803] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:19.857907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:19.857907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.017492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d986cc-657b-4cf2-9b67-4173e4f00069 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.022492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.023472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.023855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.024262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.036536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.038335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:20.054586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 897/1804] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:20.762044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:20.762044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:20.795617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803f4d1a-57f8-4b96-906c-353e96c81f81 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:20.802871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:20.803608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:20.804757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:20.805049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] PATH_INFO: `/v3/projects/3c471e6889aa4a1ea50338db56671e98` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:20.810443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c471e6889aa4a1ea50338db56671e98)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:20.857117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:21.149618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c471e6889aa4a1ea50338db56671e98', 'request_id': 'req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.156049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c471e6889aa4a1ea50338db56671e98', 'request_id': 'req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.162150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c471e6889aa4a1ea50338db56671e98', 'request_id': 'req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.181838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:21.480847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] The token cache is being invalidate because project 3c471e6889aa4a1ea50338db56671e98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:21.480847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.480847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:21.480847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 908/1805] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:20 2022] DELETE /identity/v3/projects/3c471e6889aa4a1ea50338db56671e98 => generated 0 bytes in 724 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:21.483828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:21.500425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c7d5f-dd7a-4e1b-bab5-5ce9dcebd045 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:21.565964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:21.566602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:21.566885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:21.567156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] PATH_INFO: `/v3/users/e8560b3031ec4a639f2903a64ce9a0ff` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:21.568350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8560b3031ec4a639f2903a64ce9a0ff)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:21.570287 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:22.128359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8560b3031ec4a639f2903a64ce9a0ff'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:22.130073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8560b3031ec4a639f2903a64ce9a0ff'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:22.145699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8560b3031ec4a639f2903a64ce9a0ff'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:22.178225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8560b3031ec4a639f2903a64ce9a0ff'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:22.179630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8560b3031ec4a639f2903a64ce9a0ff'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:22.330804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 898/1806] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:21 2022] DELETE /identity/v3/users/e8560b3031ec4a639f2903a64ce9a0ff => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:22.339115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:23.177964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.180719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.198856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c7dfa-6cd1-4fba-9c0f-71eefea8b164 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.276690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.277379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.277663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.277938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] PATH_INFO: `/v3/users/006d2fa12c7e4851a852475ce648d82d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.279139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=006d2fa12c7e4851a852475ce648d82d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:23.281169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:23.494682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.498130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.521282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f000079-476b-418d-9966-432113f98d70 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.529891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.531016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.531551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.532102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] PATH_INFO: `/v3/projects/300f3c5326b84a1bae47eb9f595f6063` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.534177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=300f3c5326b84a1bae47eb9f595f6063)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:23.538091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:23.570935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '006d2fa12c7e4851a852475ce648d82d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.572119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '006d2fa12c7e4851a852475ce648d82d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.581745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '006d2fa12c7e4851a852475ce648d82d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.582516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '006d2fa12c7e4851a852475ce648d82d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.596808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-553b7977-828a-4741-9690-b13ea5e6e076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '006d2fa12c7e4851a852475ce648d82d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.728937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '300f3c5326b84a1bae47eb9f595f6063', 'request_id': 'req-e16c7f82-b129-4227-bb3d-7447a0098af9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.736918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '300f3c5326b84a1bae47eb9f595f6063', 'request_id': 'req-e16c7f82-b129-4227-bb3d-7447a0098af9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.741115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 909/1807] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:23 2022] DELETE /identity/v3/users/006d2fa12c7e4851a852475ce648d82d => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:23.747118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.748119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.748726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.749102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '300f3c5326b84a1bae47eb9f595f6063', 'request_id': 'req-e16c7f82-b129-4227-bb3d-7447a0098af9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.771358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:23.772721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.868210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 910/1808] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:23.885623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:23.887509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.938006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] The token cache is being invalidate because project 300f3c5326b84a1bae47eb9f595f6063 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:23.938483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.940853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e16c7f82-b129-4227-bb3d-7447a0098af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:23.944416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 899/1809] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:23 2022] DELETE /identity/v3/projects/300f3c5326b84a1bae47eb9f595f6063 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:23.952860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5cef-6ff2-412f-91b5-92751a4731e1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.953822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5cef-6ff2-412f-91b5-92751a4731e1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.955245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f5cef-6ff2-412f-91b5-92751a4731e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.987007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd138378-c687-4717-a4a5-e81f256f2f1e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:23.991310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:23.991934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:23.992246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:23.992526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:23.994629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:23.998728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d59f5cef-6ff2-412f-91b5-92751a4731e1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.001955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d59f5cef-6ff2-412f-91b5-92751a4731e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.021362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 900/1810] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.031422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.032277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.032779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.033096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.033621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.062411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 911/1811] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:24.076904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.078119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.096193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 901/1812] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:24.099698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.110805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:24.112365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f4278b-56f9-4280-b224-12d3cd2f5272 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.120196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.121579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.122001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.122638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.123488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:24.123985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.125666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.140299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 912/1813] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:24.145002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.157261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40546eb-c5d7-478d-a039-71af2c17f3bd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.162672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.163298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.163578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.163841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.164669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.166603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.265693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 902/1814] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/projects => generated 369 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:24.269414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.295084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610c0e48-df41-467a-9ab5-d4794b382965 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.360532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.361171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.361476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.361756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.362386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.364365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.467150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 913/1815] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/users => generated 285 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:24.472350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.494038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f12525-c78a-4537-8209-da0ae7233d99 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.502386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.506311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.506311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.506311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.506311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.507230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.520462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 903/1816] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:24.525472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.542314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e371ee9-fda1-42fe-aade-d0ae3682a8f8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.550602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.551573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.551873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.552042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.552832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.554462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3957603d-3342-4646-9fae-9f9a8fd53db9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.566292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 914/1817] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:24 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:24.570580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.582063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df82aec-b41c-4d63-ba66-431c2e342a03 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.587537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.588435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.588803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.589165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] PATH_INFO: `/v3/projects/20cfe7cdb2894adda73acb6dda4252e0/users/9eba70844f944f38ad3401ad26a6574f/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.606971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20cfe7cdb2894adda73acb6dda4252e0, user_id=9eba70844f944f38ad3401ad26a6574f, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.609784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.748685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 904/1818] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:24 2022] PUT /identity/v3/projects/20cfe7cdb2894adda73acb6dda4252e0/users/9eba70844f944f38ad3401ad26a6574f/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:24.760836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.761118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.761391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.799468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] MFA Rules not processed for user `9eba70844f944f38ad3401ad26a6574f`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:24.799468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.803643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.809127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:24.899375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 915/1819] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:24.911471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:24.911727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce46afd-feda-47a4-af85-2f1821361600 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.913565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.914498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.915106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.915567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] PATH_INFO: `/v3/projects/b7c7a6ad527c48ecb999a1338f92d87e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.917172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7c7a6ad527c48ecb999a1338f92d87e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.920298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:24.925467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6c8648-2070-4a85-bf12-7b8fb9b58048 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:24.931383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:24.932409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:24.932830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:24.933240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:24.934187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:24.937286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.070313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 916/1820] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:24 2022] POST /identity/v3/projects => generated 367 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:25.077864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.097052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7c7a6ad527c48ecb999a1338f92d87e', 'request_id': 'req-da191945-6586-4313-8440-ee52a6517871'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.107129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7c7a6ad527c48ecb999a1338f92d87e', 'request_id': 'req-da191945-6586-4313-8440-ee52a6517871'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.117722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7c7a6ad527c48ecb999a1338f92d87e', 'request_id': 'req-da191945-6586-4313-8440-ee52a6517871'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.172833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.188236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9abac-407a-4a16-9280-f692d3b22359 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.194144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.194773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.195083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.195361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.195997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.197973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.282602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] The token cache is being invalidate because project b7c7a6ad527c48ecb999a1338f92d87e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:25.282932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.283630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.285302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 905/1821] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:24 2022] DELETE /identity/v3/projects/b7c7a6ad527c48ecb999a1338f92d87e => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:25.291148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.292563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.328209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 917/1822] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:25.336648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.339195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.368954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1bce54-6397-4555-bf1b-53dc777bb35c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.382144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.383146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.383635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.383971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.385090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.388295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.410189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 918/1823] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:25.411374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da191945-6586-4313-8440-ee52a6517871 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.414619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.416767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.417412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.417698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.418006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] PATH_INFO: `/v3/users/d808a4e7226b41789677b8595513798f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.419167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d808a4e7226b41789677b8595513798f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.421413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.435210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49be5dd9-e0bc-4273-ac8d-27aa898c8cba admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.443614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.444332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.444610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.444979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] PATH_INFO: `/v3/projects/c135d0e3efe54ea691df3d3332813ed7/users/b2d8c9b2a5064c769be80d5ea95afb6e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.470497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c135d0e3efe54ea691df3d3332813ed7, user_id=b2d8c9b2a5064c769be80d5ea95afb6e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.474285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.607313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:25.613651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 919/1824] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:25 2022] PUT /identity/v3/projects/c135d0e3efe54ea691df3d3332813ed7/users/b2d8c9b2a5064c769be80d5ea95afb6e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:25.617772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.665545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd808a4e7226b41789677b8595513798f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.666759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd808a4e7226b41789677b8595513798f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.675832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd808a4e7226b41789677b8595513798f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.689587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817fb8c1-b2b5-47aa-8aa7-1c798726195d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.693027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd808a4e7226b41789677b8595513798f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.693901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd808a4e7226b41789677b8595513798f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:25.694777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.695421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.695686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.695963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.696655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.698274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.711409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 920/1825] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:25 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:25.717357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:25.718338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.737126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e1e73b-620a-48bd-9ad8-f39623307dfe admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:25.745600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:25.747590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.747950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.748262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] PATH_INFO: `/v3/projects/c135d0e3efe54ea691df3d3332813ed7/users/b2d8c9b2a5064c769be80d5ea95afb6e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.752745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=c135d0e3efe54ea691df3d3332813ed7, user_id=b2d8c9b2a5064c769be80d5ea95afb6e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:25.755493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82eb401e-c4ab-4725-945a-8943bb9939fa admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:25.771052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 921/1826] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:25 2022] PUT /identity/v3/projects/c135d0e3efe54ea691df3d3332813ed7/users/b2d8c9b2a5064c769be80d5ea95afb6e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:25.782130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:25.782349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:25.782609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:25.826208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] MFA Rules not processed for user `b2d8c9b2a5064c769be80d5ea95afb6e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:25.827758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:25.861037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 906/1827] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:25 2022] DELETE /identity/v3/users/d808a4e7226b41789677b8595513798f => generated 0 bytes in 565 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:25.980329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 922/1828] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:25 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:26.006468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.008306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.100433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b272cd81-d740-428a-b5dc-d708cfe610d6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.105010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.107651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.107651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.107651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.204707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.208184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.232206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 907/1829] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:26.235361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.239017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.332189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee47f30-acf9-494d-a71e-3f29d116f728 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.428712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:26.431895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.461818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f428f5-d866-49d5-9033-c05ff7ccce1c service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:26.464496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.466200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.466861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.467623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] PATH_INFO: `/v3/users/d985abff33e4413c87709f1d83c0fd0b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.469861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=d985abff33e4413c87709f1d83c0fd0b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.470599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:26.472794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:26.473315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:26.474310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:26.474966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.479903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:26.583330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:26.587305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78958d0-df8e-4ec0-969c-e0dd8e540dfa service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:26.612396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 908/1830] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:26 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:26.838275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd985abff33e4413c87709f1d83c0fd0b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.839062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd985abff33e4413c87709f1d83c0fd0b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.848740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd985abff33e4413c87709f1d83c0fd0b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.849607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd985abff33e4413c87709f1d83c0fd0b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:26.865195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd985abff33e4413c87709f1d83c0fd0b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.052386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 923/1831] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:26 2022] DELETE /identity/v3/users/d985abff33e4413c87709f1d83c0fd0b => generated 0 bytes in 817 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:27.056725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.057018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.057297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.092694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:27.093669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.209084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 909/1832] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:27.222482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.231916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.303716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e250b543-622e-4ef8-b228-303a0d444c1a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.307657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.308933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.309233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.309506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.320535 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.321899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.338623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 924/1833] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:27.373791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.375482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:27.409445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52255e21-63c1-4466-9f0d-366770be9c48 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:27.483849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:27.483849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:27.483849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:27.483849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] PATH_INFO: `/v3/projects/20a2a3659dab4a83852fa9442758f4b8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:27.485037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] RBAC: Authorizing `identity:delete_project(project_id=20a2a3659dab4a83852fa9442758f4b8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:27.485037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:27.622532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:27.761119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20a2a3659dab4a83852fa9442758f4b8', 'request_id': 'req-c0e4512c-02b2-433d-b11f-5cfcef09f98e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.780219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20a2a3659dab4a83852fa9442758f4b8', 'request_id': 'req-c0e4512c-02b2-433d-b11f-5cfcef09f98e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.798398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20a2a3659dab4a83852fa9442758f4b8', 'request_id': 'req-c0e4512c-02b2-433d-b11f-5cfcef09f98e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.935549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] The token cache is being invalidate because project 20a2a3659dab4a83852fa9442758f4b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:27.936344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.937094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c0e4512c-02b2-433d-b11f-5cfcef09f98e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:27.941052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 910/1834] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:27 2022] DELETE /identity/v3/projects/20a2a3659dab4a83852fa9442758f4b8 => generated 0 bytes in 570 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:27.948098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:27.978733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424f22a5-145d-4f87-a08c-91deb7f02a87 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.101735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.102555 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.103041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.103541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] PATH_INFO: `/v3/users/e1dd5839a2b6495e90938ef6852a8d5b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.105205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1dd5839a2b6495e90938ef6852a8d5b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.109484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.254934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9e69-f401-4ff9-b159-3c8864d56bc3 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.255447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9e69-f401-4ff9-b159-3c8864d56bc3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.255940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2e9e69-f401-4ff9-b159-3c8864d56bc3 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.258549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 911/1835] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:02:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:02:28.280427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.280911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.282185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.334713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:28.341246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.384778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1dd5839a2b6495e90938ef6852a8d5b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.385508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1dd5839a2b6495e90938ef6852a8d5b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.396576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1dd5839a2b6495e90938ef6852a8d5b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.397267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1dd5839a2b6495e90938ef6852a8d5b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.418220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1dd5839a2b6495e90938ef6852a8d5b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.438624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 912/1836] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:02:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:28.455546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.459949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.507032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a46cd4-1263-47b5-8ed0-04f8cae1f916 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.510985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.512299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.512802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.519538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] PATH_INFO: `/v3/projects/785e883df3fe4eda83165b09544c68b0` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.519538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] RBAC: Authorizing `identity:delete_project(project_id=785e883df3fe4eda83165b09544c68b0)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.519538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.535080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 925/1837] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:27 2022] DELETE /identity/v3/users/e1dd5839a2b6495e90938ef6852a8d5b => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:28.536032 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.539034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:28.657706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee79534-dfe0-45a5-9a41-3415617cbde3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.677634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785e883df3fe4eda83165b09544c68b0', 'request_id': 'req-71d7373d-8265-4507-b76f-26ac6c58513c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.689295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785e883df3fe4eda83165b09544c68b0', 'request_id': 'req-71d7373d-8265-4507-b76f-26ac6c58513c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.700267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785e883df3fe4eda83165b09544c68b0', 'request_id': 'req-71d7373d-8265-4507-b76f-26ac6c58513c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.764019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.764860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.765076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.765602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:28.775682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:28.777871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:28.789394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 926/1838] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:28.817684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] The token cache is being invalidate because project 785e883df3fe4eda83165b09544c68b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:28.817990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.818912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:28.821125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 913/1839] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:28 2022] DELETE /identity/v3/projects/785e883df3fe4eda83165b09544c68b0 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:28.826327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:28.959300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c1372f-0b6b-434d-aa31-979c91ea4df9 service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:28.986787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:28.987453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:28.992679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:28.993628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] PATH_INFO: `/v3/users/b625b6c0e0894d208fa14b564c7e20d2` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:29.009119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b625b6c0e0894d208fa14b564c7e20d2)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:29.019056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:29.533870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b625b6c0e0894d208fa14b564c7e20d2'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.535026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b625b6c0e0894d208fa14b564c7e20d2'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.545330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b625b6c0e0894d208fa14b564c7e20d2'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.547234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b625b6c0e0894d208fa14b564c7e20d2'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.571505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b66c7a4b-01d0-48da-a505-875088e9cb63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b625b6c0e0894d208fa14b564c7e20d2'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:29.904707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 927/1840] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:28 2022] DELETE /identity/v3/users/b625b6c0e0894d208fa14b564c7e20d2 => generated 0 bytes in 1079 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:29.914031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:29.914031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:29.939857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d7373d-8265-4507-b76f-26ac6c58513c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.006222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.007393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.007943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.008473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] PATH_INFO: `/v3/projects/70c4454491554247a58be4b95f5c693c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.010802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=70c4454491554247a58be4b95f5c693c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.013007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.403561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '70c4454491554247a58be4b95f5c693c', 'request_id': 'req-29c6ed8e-6975-4554-aef9-e76c37660e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.423744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '70c4454491554247a58be4b95f5c693c', 'request_id': 'req-29c6ed8e-6975-4554-aef9-e76c37660e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.440848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '70c4454491554247a58be4b95f5c693c', 'request_id': 'req-29c6ed8e-6975-4554-aef9-e76c37660e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.541748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] The token cache is being invalidate because project 70c4454491554247a58be4b95f5c693c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:30.543768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.545272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-29c6ed8e-6975-4554-aef9-e76c37660e3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:30.551548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 914/1841] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:29 2022] DELETE /identity/v3/projects/70c4454491554247a58be4b95f5c693c => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:30.560964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.561677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.561677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.591670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:30.593116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.675282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 928/1842] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:30.684223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.684578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.684860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.708584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:30.710083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:30.727455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:30.730698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 915/1843] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:30.737172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.775226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223e87de-9792-46f9-a3a2-544609b29a17 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.788586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.791780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.792342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.792800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.794389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:30.795294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.798100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:30.815938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 929/1844] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:30.822740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:30.843563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dcbc33-b2e5-4cf0-aa5e-3ba6b7968e31 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:30.855353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:30.857075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:30.857906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:30.859663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:30.861548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:30.865987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.035717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 916/1845] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:30 2022] POST /identity/v3/projects => generated 385 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:31.036996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.037694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.038816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:31.051479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33572e6-242b-425b-83cf-ec3fbaf7feca admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.129513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9512ce-4c57-4a70-b172-ab1d0a4308a8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.135214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.135873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.140392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.140392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] PATH_INFO: `/v3/projects/e48d95dff0b548f984e70cb884027967` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.140392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e48d95dff0b548f984e70cb884027967)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.140392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.141905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.142731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.143094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.143436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.144194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.147133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.288790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 930/1846] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/users => generated 293 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:31.295821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.317850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2d5800-72e1-4c69-b06b-90d505766a0c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.351215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e48d95dff0b548f984e70cb884027967', 'request_id': 'req-e0051501-15da-473e-ada1-d1de35d8c3a6'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:31.368719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e48d95dff0b548f984e70cb884027967', 'request_id': 'req-e0051501-15da-473e-ada1-d1de35d8c3a6'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:31.382889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e48d95dff0b548f984e70cb884027967', 'request_id': 'req-e0051501-15da-473e-ada1-d1de35d8c3a6'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:31.421457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.422406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.422686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.423007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.423670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.425491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.438452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 931/1847] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:31.442544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.453770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dab16cd-817b-4862-bc94-fdf091727584 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.459411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.460028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.460343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.460624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.461303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.462921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.475558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 932/1848] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:31.482975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.483259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] The token cache is being invalidate because project e48d95dff0b548f984e70cb884027967 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:31.483259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:31.483745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e0051501-15da-473e-ada1-d1de35d8c3a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:31.485613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 917/1849] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:31 2022] DELETE /identity/v3/projects/e48d95dff0b548f984e70cb884027967 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:31.490280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e953872-d12f-4149-bc25-dffbc49f4e37 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.490565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e953872-d12f-4149-bc25-dffbc49f4e37 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.490879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e953872-d12f-4149-bc25-dffbc49f4e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.496759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7168b6-326b-4295-ad7e-ae46327fd409 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.501812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.502487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.502767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.503045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] PATH_INFO: `/v3/projects/589c3a4b74a24763a52835b356c8913c/users/999b70b12f744658bf817cba254919f8/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.518279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0e953872-d12f-4149-bc25-dffbc49f4e37 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:31.519509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0e953872-d12f-4149-bc25-dffbc49f4e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:31.523397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=589c3a4b74a24763a52835b356c8913c, user_id=999b70b12f744658bf817cba254919f8, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.523623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25f9e37-a284-472a-b52a-080b6fbbeed8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.531673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 918/1850] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:31.539741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.540031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.540542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.565561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:31.566817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:31.575741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 919/1851] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:31.580281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.591459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071a7d4-5267-4d8c-968e-1a22bfdd8f7d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.596875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.597534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.597829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.598114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.598783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:31.599087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.600762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.614720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 920/1852] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:31.619598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.632174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dd60fd-d1f1-4fc9-9dea-2829a1c24f44 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.638313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.638938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.639218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.639494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.640172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.642130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.656015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 933/1853] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:31 2022] PUT /identity/v3/projects/589c3a4b74a24763a52835b356c8913c/users/999b70b12f744658bf817cba254919f8/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:31.674376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.674728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.675058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.709241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] MFA Rules not processed for user `999b70b12f744658bf817cba254919f8`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:31.710603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:31.785322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 921/1854] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/projects => generated 383 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:31.787828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 934/1855] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:31.789803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.799437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:31.800937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:31.875779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b64f8-6ebf-405d-b7d2-34d7117a38d6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.882863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc89405a-c3e3-4ee6-9211-0399e8e82375 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:31.884430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.885060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.885500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.885675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.886294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.888815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:31.889802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:31.890651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:31.891022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:31.891305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:31.891971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:31.893944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.076274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 922/1856] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/users => generated 292 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:32.079762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.099651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a33ccd-8169-4bab-b25a-db4138528d8b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.109646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.110719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.111431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.111913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.113121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.116060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.133411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 923/1857] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:32.138094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.149766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53069ece-607e-42f2-abf0-d60f911ef966 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.151325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 935/1858] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:31 2022] POST /identity/v3/projects => generated 387 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:32.158183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.217778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.218439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.219202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.219202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.219972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.221739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.246913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 924/1859] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:32.252949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.270117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7be93-a822-44a5-9607-5404534e5441 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.275847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.276676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.277506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.277939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] PATH_INFO: `/v3/projects/0055fb3d13b545e4b33b8ab3cee9053b/users/9d8e6ce3f0944df889fb97a21bd451a7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.287322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c30fea-b55c-437f-abbb-759037e04d1b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.296031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0055fb3d13b545e4b33b8ab3cee9053b, user_id=9d8e6ce3f0944df889fb97a21bd451a7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.296520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.297780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.298409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.298873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e93259e-1e3a-402b-899f-22eed369fb5d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.299139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.300494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.305111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.393930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:32.448186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 936/1860] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/users => generated 300 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:32.469835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.478133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d149be-d4f4-4de7-8da1-9dff2d32b546 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.478434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 925/1861] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:32 2022] PUT /identity/v3/projects/0055fb3d13b545e4b33b8ab3cee9053b/users/9d8e6ce3f0944df889fb97a21bd451a7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:32.491095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.492101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.492846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.546996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] MFA Rules not processed for user `9d8e6ce3f0944df889fb97a21bd451a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:32.549513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:32.579967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.580814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.581017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.581274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.582084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.584438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.609697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 937/1862] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:32.614905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.629557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830e4f29-9394-4c34-8962-b2ba424e3b48 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.635686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.637746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.638151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.638509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] PATH_INFO: `/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3/users/ebf983b56f0d4d8b8667824a7e91b78c/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.660011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4afe1c4b6c4df1a72fbbf796f007f3, user_id=ebf983b56f0d4d8b8667824a7e91b78c, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.663020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.692994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 926/1863] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:32.699953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.717805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98c9aaa-c109-480a-b7ab-9596ec1b1da3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:32.734669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:32.735952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:32.736506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:32.737097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:32.738354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:32.742895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:32.857279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:32.867805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 938/1864] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:32 2022] PUT /identity/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3/users/ebf983b56f0d4d8b8667824a7e91b78c/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:32.873908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.933247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 927/1865] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/projects => generated 383 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:32.942927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:32.987763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69957af-963d-43c3-afbe-dc49d3293092 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.097827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.099850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.100478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.100827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.103564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.103746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.135065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa2e9d0-be4e-4bc2-b862-893349214084 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.146376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 939/1866] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:32 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:33.148493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.150168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.150946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.151244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.151525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.152779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.158618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.179680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b03eb9-5bd5-45b6-9b87-f09ce9898620 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.199425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.202750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.203367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.205502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] PATH_INFO: `/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3/users/ebf983b56f0d4d8b8667824a7e91b78c/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.209519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a4afe1c4b6c4df1a72fbbf796f007f3, user_id=ebf983b56f0d4d8b8667824a7e91b78c, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.219236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30308a18-c966-4881-b7fe-f0ca9bb9ba9f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.239018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 940/1867] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3/users/ebf983b56f0d4d8b8667824a7e91b78c/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:33.257914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.258490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.262989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.276084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 928/1868] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:32 2022] POST /identity/v3/users => generated 298 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:33.279269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.323154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] MFA Rules not processed for user `ebf983b56f0d4d8b8667824a7e91b78c`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:33.325404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:33.381431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3dab98-82e7-43e0-943a-6e5b6de2867f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.502181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.503696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.504157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.504714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.507026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.516796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.540217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 929/1869] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.546710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.554127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 941/1870] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:33 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:33.572874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e30656-a76e-44b5-a70f-5c6f6acd5987 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.586358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.588258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.588763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.589323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] PATH_INFO: `/v3/projects/f836c35943b34e70ab4ff865b3255fd7/users/7b3468a537ea4330b0bb09839c90ff71/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.631862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f836c35943b34e70ab4ff865b3255fd7, user_id=7b3468a537ea4330b0bb09839c90ff71, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.635741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.683811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.685583 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:33.749192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 930/1871] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/f836c35943b34e70ab4ff865b3255fd7/users/7b3468a537ea4330b0bb09839c90ff71/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:33.755660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:33.791405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af7812d-222b-421e-9de3-0690a30dd26f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.921019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.921658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.921941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.922215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.938426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c346b08-e8ad-4cf9-8952-747b20b08b0c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:33.956826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:33.957089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:33.961979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:33.962880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:33.965155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:33.969367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:33.993929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 931/1872] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:33.999080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:34.021674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0feb7c61-6690-46c9-b2d7-ed0106c552de admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:34.034482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:34.036398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.038675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.038675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] PATH_INFO: `/v3/projects/f836c35943b34e70ab4ff865b3255fd7/users/7b3468a537ea4330b0bb09839c90ff71/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.039713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f836c35943b34e70ab4ff865b3255fd7, user_id=7b3468a537ea4330b0bb09839c90ff71, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:34.042588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff87637-fc5f-47f0-a566-b4cc76eeb301 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:34.055220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:34.056900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:34.059579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 932/1873] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:33 2022] PUT /identity/v3/projects/f836c35943b34e70ab4ff865b3255fd7/users/7b3468a537ea4330b0bb09839c90ff71/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:34.081605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.081981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 942/1874] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:33 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:34.084448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.085006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.125210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] MFA Rules not processed for user `7b3468a537ea4330b0bb09839c90ff71`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:34.126970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:34.204933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 933/1875] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:34.271171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:34.276669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:34.278442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:34.292292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c375c9a-88a9-4d95-b43b-e83cfe78a80e service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:34.362798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb33f8-d05b-42a3-9c7c-331db700bb17 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:34.367378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:34.368094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.368420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.368687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.369788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:34.370441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.371611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.371897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] PATH_INFO: `/v3/users/9eba70844f944f38ad3401ad26a6574f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.373227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9eba70844f944f38ad3401ad26a6574f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:34.375377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:34.383193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:34.384836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:34.395543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 943/1876] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:34.507052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:34.516905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d93d08-d9c9-4d97-b1a0-a9abc53db807 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:34.522847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:34.523945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.524546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.525131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.606976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:34.609012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebd924b-7777-4eb9-8958-8560ddedbfd1 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:34.620504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 944/1877] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:34 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:34.700734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9eba70844f944f38ad3401ad26a6574f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:34.701475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9eba70844f944f38ad3401ad26a6574f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:34.718414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9eba70844f944f38ad3401ad26a6574f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:34.772663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9eba70844f944f38ad3401ad26a6574f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:34.775527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9eba70844f944f38ad3401ad26a6574f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:34.927395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 934/1878] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:34 2022] DELETE /identity/v3/users/9eba70844f944f38ad3401ad26a6574f => generated 0 bytes in 651 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:34.932925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:34.933486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:34.933486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:34.972687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:34.974026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:35.099418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 945/1879] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:35.113081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:35.114847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:35.209193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:35.301065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20412c5-198d-4bd7-b88d-0c7c36f2ecf2 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:35.305852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:35.306511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:35.307012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:35.307360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:35.322662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:35.324373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:35.343402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 935/1880] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:36.050163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:36.054532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:36.076054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b35470-fb0a-48b6-92e9-fc4a62e3ec03 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:36.089282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:36.089282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:36.089282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:36.089282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] PATH_INFO: `/v3/projects/20cfe7cdb2894adda73acb6dda4252e0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:36.090118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] RBAC: Authorizing `identity:delete_project(project_id=20cfe7cdb2894adda73acb6dda4252e0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:36.090823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:36.140835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:36.142291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:36.157647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff3aa79-2274-4796-b6ae-2d37d78a2fd0 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:36.163912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:36.166381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:36.166381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:36.166381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] PATH_INFO: `/v3/users/999b70b12f744658bf817cba254919f8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:36.166812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=999b70b12f744658bf817cba254919f8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:36.173395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:36.254495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20cfe7cdb2894adda73acb6dda4252e0', 'request_id': 'req-2578ac33-9508-4400-aea8-1d710b7f780f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.268395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20cfe7cdb2894adda73acb6dda4252e0', 'request_id': 'req-2578ac33-9508-4400-aea8-1d710b7f780f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.282884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20cfe7cdb2894adda73acb6dda4252e0', 'request_id': 'req-2578ac33-9508-4400-aea8-1d710b7f780f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.440408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] The token cache is being invalidate because project 20cfe7cdb2894adda73acb6dda4252e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:36.441266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.443483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.446662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 946/1881] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:36 2022] DELETE /identity/v3/projects/20cfe7cdb2894adda73acb6dda4252e0 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:36.454453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:36.462683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:36.477927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '999b70b12f744658bf817cba254919f8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.479344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '999b70b12f744658bf817cba254919f8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.492068 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '999b70b12f744658bf817cba254919f8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.550232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '999b70b12f744658bf817cba254919f8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.551376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-47e51c0e-5193-4530-b296-130c85ea7a18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '999b70b12f744658bf817cba254919f8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.603133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2578ac33-9508-4400-aea8-1d710b7f780f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:36.608369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:36.609043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:36.609321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:36.609597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] PATH_INFO: `/v3/users/b2d8c9b2a5064c769be80d5ea95afb6e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:36.610834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2d8c9b2a5064c769be80d5ea95afb6e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:36.612823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:36.696866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:36.725266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 936/1882] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:36 2022] DELETE /identity/v3/users/999b70b12f744658bf817cba254919f8 => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:36.731969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:36.732755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:36.733185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:36.756762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:36.758407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:36.866797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 937/1883] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:36.891700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:36.904289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:36.914865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2d8c9b2a5064c769be80d5ea95afb6e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.916715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2d8c9b2a5064c769be80d5ea95afb6e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.950389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2d8c9b2a5064c769be80d5ea95afb6e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.953403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2d8c9b2a5064c769be80d5ea95afb6e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:36.999379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2d8c9b2a5064c769be80d5ea95afb6e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:37.041661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce57b7a-b848-401f-bd47-1f9e00eaab27 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:37.046385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:37.047159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:37.047578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:37.047994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:37.050474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:37.065538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:37.067060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:37.079250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 938/1884] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:37.116432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 947/1885] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:36 2022] DELETE /identity/v3/users/b2d8c9b2a5064c769be80d5ea95afb6e => generated 0 bytes in 663 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:37.827764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:37.829665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:37.846110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894c8572-c918-4ae6-b8bb-91a3749a67bf service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:37.913401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:37.914123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:37.914452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:37.914769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] PATH_INFO: `/v3/projects/589c3a4b74a24763a52835b356c8913c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:37.916021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=589c3a4b74a24763a52835b356c8913c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:37.918687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:38.134108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '589c3a4b74a24763a52835b356c8913c', 'request_id': 'req-62cb1fd4-5240-4680-8f92-5e3336adfad5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.139326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:38.146974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '589c3a4b74a24763a52835b356c8913c', 'request_id': 'req-62cb1fd4-5240-4680-8f92-5e3336adfad5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.158365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '589c3a4b74a24763a52835b356c8913c', 'request_id': 'req-62cb1fd4-5240-4680-8f92-5e3336adfad5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.159078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:38.162365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.201158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e01f8c-8f90-49e1-9e77-f4d9d550e306 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:38.271954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] The token cache is being invalidate because project 589c3a4b74a24763a52835b356c8913c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:38.272286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.278453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.282273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 939/1886] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:37 2022] DELETE /identity/v3/projects/589c3a4b74a24763a52835b356c8913c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:38.288927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:38.293819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.309997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:38.310871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.311123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.311497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] PATH_INFO: `/v3/projects/c135d0e3efe54ea691df3d3332813ed7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.312818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c135d0e3efe54ea691df3d3332813ed7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:38.315476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:38.417399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb1fd4-5240-4680-8f92-5e3336adfad5 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:38.422147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:38.422821 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.423101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.423378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] PATH_INFO: `/v3/users/ebf983b56f0d4d8b8667824a7e91b78c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.424577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf983b56f0d4d8b8667824a7e91b78c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:38.426514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:38.465867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c135d0e3efe54ea691df3d3332813ed7', 'request_id': 'req-930826bf-8279-4e10-ad6f-5944b4aa1454'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.473795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c135d0e3efe54ea691df3d3332813ed7', 'request_id': 'req-930826bf-8279-4e10-ad6f-5944b4aa1454'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.481313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c135d0e3efe54ea691df3d3332813ed7', 'request_id': 'req-930826bf-8279-4e10-ad6f-5944b4aa1454'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.585791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] The token cache is being invalidate because project c135d0e3efe54ea691df3d3332813ed7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:38.586178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.586945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-930826bf-8279-4e10-ad6f-5944b4aa1454 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.588756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 948/1887] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:38 2022] DELETE /identity/v3/projects/c135d0e3efe54ea691df3d3332813ed7 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:38.597038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa3a2-c6e5-47ff-ab04-e07c14a3d573 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.597549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa3a2-c6e5-47ff-ab04-e07c14a3d573 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.598096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234fa3a2-c6e5-47ff-ab04-e07c14a3d573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.624548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-234fa3a2-c6e5-47ff-ab04-e07c14a3d573 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:38.625844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-234fa3a2-c6e5-47ff-ab04-e07c14a3d573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.680528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf983b56f0d4d8b8667824a7e91b78c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.683211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf983b56f0d4d8b8667824a7e91b78c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.701025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 949/1888] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:38.709760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.709971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.710239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.711095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf983b56f0d4d8b8667824a7e91b78c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.733038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:38.734273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:38.738251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf983b56f0d4d8b8667824a7e91b78c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.739431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf983b56f0d4d8b8667824a7e91b78c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:38.744429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 950/1889] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:38.748208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:38.757635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24cd2a5-0dab-4ef8-9048-3513fa316c07 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:38.763615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:38.764754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.764754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.765302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.765624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:38.765932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:38.767588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:38.784502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 951/1890] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:38.789155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:38.809379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afe6317-927c-418b-aab2-be7b5c3d0f7c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:38.812203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:38.812837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:38.813116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:38.813699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:38.814362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:38.816395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:38.836918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 940/1891] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:38 2022] DELETE /identity/v3/users/ebf983b56f0d4d8b8667824a7e91b78c => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:38.956100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 952/1892] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:38 2022] POST /identity/v3/projects => generated 387 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:38.957401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.026491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b221aa1-d590-4a52-8eb1-09bbc11ed0f6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.111574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.112127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.112449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.112849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.113390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.115463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.287892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 941/1893] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:38 2022] POST /identity/v3/users => generated 294 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:39.297114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.308759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907c49e6-506d-4dcf-a53c-42c0d0872548 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.321893 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.322565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.323028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.323299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.324052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.325627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.341838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 953/1894] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:39.345446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.358980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4a0f7a-66c8-46be-b3b3-29d55a5046ea admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.388782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.389552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.389843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.390122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.390958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.392968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.410257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 942/1895] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:39.413970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.437335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27150e1-12cc-4b3a-9453-b73d3894330b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.446563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.450261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.450888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.451491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] PATH_INFO: `/v3/projects/ab9d5ae6b1674054ae040aeab75d31fd/users/5ea2ea4f2e154961871de40cbfe9cea0/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.481425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab9d5ae6b1674054ae040aeab75d31fd, user_id=5ea2ea4f2e154961871de40cbfe9cea0, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.485150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0580c5-0463-4e8d-a816-81c6b7cd1dbe admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.634800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.638792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:39.643327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 954/1896] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:39 2022] PUT /identity/v3/projects/ab9d5ae6b1674054ae040aeab75d31fd/users/5ea2ea4f2e154961871de40cbfe9cea0/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:39.656278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.656821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.656963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:39.657525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.662908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3698f7af-b4be-43c4-834f-36ecd91b1c6b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.695976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] MFA Rules not processed for user `5ea2ea4f2e154961871de40cbfe9cea0`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:39.697417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:39.764339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.765543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.765985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.766585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] PATH_INFO: `/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.768201 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a4afe1c4b6c4df1a72fbbf796f007f3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.771726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.796793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 955/1897] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:39.799866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:39.814686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0d5d20-ae4e-4586-af79-dc570fdcd2d6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:39.820118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:39.820844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:39.821154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:39.821441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:39.822528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:39.825478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:39.932590 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a4afe1c4b6c4df1a72fbbf796f007f3', 'request_id': 'req-295457d4-817f-4648-84ec-d1afded39a30'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:39.946073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a4afe1c4b6c4df1a72fbbf796f007f3', 'request_id': 'req-295457d4-817f-4648-84ec-d1afded39a30'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:39.956614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a4afe1c4b6c4df1a72fbbf796f007f3', 'request_id': 'req-295457d4-817f-4648-84ec-d1afded39a30'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:39.985158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 956/1898] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:39 2022] POST /identity/v3/projects => generated 387 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:39.991317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.081311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad977b34-4838-4011-b279-bd3f9eeecf42 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.086505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.087126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.087501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.088756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.088756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.091987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.099311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] The token cache is being invalidate because project 9a4afe1c4b6c4df1a72fbbf796f007f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:40.099805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.101138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-295457d4-817f-4648-84ec-d1afded39a30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:40.104555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 943/1899] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:39 2022] DELETE /identity/v3/projects/9a4afe1c4b6c4df1a72fbbf796f007f3 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:40.111286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32432a15-d273-4afe-a20b-430875328faa None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.111813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32432a15-d273-4afe-a20b-430875328faa None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.112525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32432a15-d273-4afe-a20b-430875328faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.138692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-32432a15-d273-4afe-a20b-430875328faa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:40.139959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-32432a15-d273-4afe-a20b-430875328faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.148916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 944/1900] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:40.157490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.157778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.158057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.181660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:40.183317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.199716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 945/1901] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:40.201477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.213166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b1f7c-404e-43dc-bcaf-4f2f121e9996 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.219299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.220087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.220549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.220902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.221659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:40.222056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.223892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.239495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 946/1902] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:40.244916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.252518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 957/1903] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:39 2022] POST /identity/v3/users => generated 300 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:40.254279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.256429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.257525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8933b-41eb-4ff1-bcb9-33103f3aec07 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.263323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.264188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.264487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.264771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.265481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.267517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.270804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c4f185-7ffe-44bc-9f64-bfce6f24fc3d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.278458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.279095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.279373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.279643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.282514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.282514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.315606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 958/1904] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:40.316661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.348889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c61d199-2920-4b9b-b1e2-8ced0c829694 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.361044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.362496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.363207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.363871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] PATH_INFO: `/v3/projects/72058380eb344ac2aa055eb04f4e3e48/users/87f5294814d54b21abaf3abca54ecd6f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.397982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72058380eb344ac2aa055eb04f4e3e48, user_id=87f5294814d54b21abaf3abca54ecd6f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.403207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.421461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:40.440123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 947/1905] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/projects => generated 381 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:40.446399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.509825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 959/1906] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:40 2022] PUT /identity/v3/projects/72058380eb344ac2aa055eb04f4e3e48/users/87f5294814d54b21abaf3abca54ecd6f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:40.516469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.745071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3981611-fa89-42ba-8356-ebccedf9a97f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.762413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83c9c9-cac5-41e6-bfbd-051aa4293089 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.771802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.772886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.773387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.773883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.775081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.778157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.778821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.778993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.779243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.779551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.780363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.782261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.807330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 960/1907] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:40.810792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.818035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:40.855649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c1e3-e8c0-4c59-a21d-acdc61c3ca8c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.861895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.864360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.864817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.865263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] PATH_INFO: `/v3/users/9d8e6ce3f0944df889fb97a21bd451a7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.868315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8e6ce3f0944df889fb97a21bd451a7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.878031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.942086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 948/1908] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:40 2022] POST /identity/v3/users => generated 291 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:40.943227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:40.957512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7719d8f4-fea7-4223-9c57-356abe4cd2c5 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:40.964624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:40.969003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:40.969003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:40.969003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] PATH_INFO: `/v3/projects/72058380eb344ac2aa055eb04f4e3e48/users/87f5294814d54b21abaf3abca54ecd6f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:40.969412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=72058380eb344ac2aa055eb04f4e3e48, user_id=87f5294814d54b21abaf3abca54ecd6f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:40.972408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:40.990739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 949/1909] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:40 2022] PUT /identity/v3/projects/72058380eb344ac2aa055eb04f4e3e48/users/87f5294814d54b21abaf3abca54ecd6f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:40.991826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.119236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7adf2-746c-449c-ae46-7f2eeb91d6cd admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.125007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.125737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.126085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.126440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.127208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.129032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bcf8b3-8445-4d63-92b6-fd1c8839d648 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.145100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 950/1910] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:41.146670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.147067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.147606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.189557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8e6ce3f0944df889fb97a21bd451a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.189864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] MFA Rules not processed for user `87f5294814d54b21abaf3abca54ecd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.190900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8e6ce3f0944df889fb97a21bd451a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.192063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.204858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8e6ce3f0944df889fb97a21bd451a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.205845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8e6ce3f0944df889fb97a21bd451a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.226041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8e6ce3f0944df889fb97a21bd451a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:41.301644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 951/1911] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.302638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.304056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.331052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 961/1912] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:40 2022] DELETE /identity/v3/users/9d8e6ce3f0944df889fb97a21bd451a7 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:41.332746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.334421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.351327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e1c45f-d0a7-4feb-a260-24205385b124 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.409751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdf2ec5-4c34-4e00-8fcb-fdb7d728daa6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.415676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.417161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.417831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.418200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.476690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.477386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.477697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.478283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.479460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.482158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a06087-355d-41ff-abca-5b298f349ab3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.502087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 952/1913] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:41.504224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.509696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.509696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.528869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.532196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.537679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.539040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.547669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 953/1914] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.551935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.552097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 962/1915] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:41.562185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.565263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.575213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498acb7-0e33-4acf-a8dd-f09c798f3aa1 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.588640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.589948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.590426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.590907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] PATH_INFO: `/v3/projects/cd5c3596bc744bf49025eed370970c42/users/cd5e059cd4054224ab179ac0e6253d1d/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.625157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd5c3596bc744bf49025eed370970c42, user_id=cd5e059cd4054224ab179ac0e6253d1d, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.629440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.704248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad43e7-b077-4133-a7e4-fcc1eccdfcd0 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.710317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.712733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.713269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.713853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.736932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.738582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd1de4-2fbe-4981-ab4d-cece833610a1 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.754202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 954/1916] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:41 2022] PUT /identity/v3/projects/cd5c3596bc744bf49025eed370970c42/users/cd5e059cd4054224ab179ac0e6253d1d/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:41.755889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.758732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.771954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 963/1917] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:41.775495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.775943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.776403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.819642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] MFA Rules not processed for user `cd5e059cd4054224ab179ac0e6253d1d`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:41.821094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:41.880188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2b60-8a0a-4dfc-8f1b-4bd7d74bac26 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.885335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.885987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.886306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.887259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] PATH_INFO: `/v3/users/5ea2ea4f2e154961871de40cbfe9cea0` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.888349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ea2ea4f2e154961871de40cbfe9cea0)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.890348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:41.917316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 964/1918] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:41.919512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:41.931883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ba9ee-e7e7-4142-865a-e417f0b83748 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:41.937495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:41.938131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:41.938673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:41.941003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:41.941003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:41.941701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.158970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 965/1919] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:41 2022] POST /identity/v3/projects => generated 379 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:42.160641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.253765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bd3e35-e51e-4416-9f73-b968bd0b0f4e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.260614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.261249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.261601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.261883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.262544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.264565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.304542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ea2ea4f2e154961871de40cbfe9cea0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:42.305323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ea2ea4f2e154961871de40cbfe9cea0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:42.319013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ea2ea4f2e154961871de40cbfe9cea0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:42.339145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ea2ea4f2e154961871de40cbfe9cea0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:42.340495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3bd1535d-f57a-4ce1-8547-96840053deb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ea2ea4f2e154961871de40cbfe9cea0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:42.362698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:42.520066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 966/1920] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:42 2022] POST /identity/v3/users => generated 296 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:42.531405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.547332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.566171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 955/1921] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:41 2022] DELETE /identity/v3/users/5ea2ea4f2e154961871de40cbfe9cea0 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:42.570950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.571728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.572632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.575053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1b760c-7d70-43c3-acfa-690d583f835a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.631073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:42.631750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.715707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 956/1922] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:42.719275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.719275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.719275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.719275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.719647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.721161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.727423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.739564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.739734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 967/1923] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:42.741080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.755780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1579663-e2b2-4338-ae97-61043b841da6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.761034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.761868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.762223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.762526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] PATH_INFO: `/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294/users/8a5ba8f0eddf4df89d8110f6a209602b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.790427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe5d40f5ecf4551acb03bf43c4b2294, user_id=8a5ba8f0eddf4df89d8110f6a209602b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.790427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.807681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ee919-f007-4c72-881b-c2c9b5a0a53b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.812457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.813207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.813854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.814198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.840131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.841738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.863899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 957/1924] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:42.872559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:42.872559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:42.893421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c65f0-d481-4976-b733-c556a18695fc service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:42.901953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:42.903631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:42.904089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:42.904402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] PATH_INFO: `/v3/projects/0055fb3d13b545e4b33b8ab3cee9053b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:42.905854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0055fb3d13b545e4b33b8ab3cee9053b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:42.907630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:42.939208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 968/1925] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:42 2022] PUT /identity/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294/users/8a5ba8f0eddf4df89d8110f6a209602b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:42.946183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.063132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2b8f6e-229e-4a25-ac09-6a1c800658ae admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.069039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:43.069818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.070184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.070616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.071424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:43.073936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:43.100215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 969/1926] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:43.102455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0055fb3d13b545e4b33b8ab3cee9053b', 'request_id': 'req-864749f0-15ae-4486-a053-c21ef2272a9d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.103404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.112180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0055fb3d13b545e4b33b8ab3cee9053b', 'request_id': 'req-864749f0-15ae-4486-a053-c21ef2272a9d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.124228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0055fb3d13b545e4b33b8ab3cee9053b', 'request_id': 'req-864749f0-15ae-4486-a053-c21ef2272a9d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.139602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4894744-3302-4631-a0dd-68db839292f9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.277034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:43.278210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] The token cache is being invalidate because project 0055fb3d13b545e4b33b8ab3cee9053b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:43.278769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.279178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.280124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.280541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.281046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] PATH_INFO: `/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294/users/8a5ba8f0eddf4df89d8110f6a209602b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.282576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 958/1927] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:42 2022] DELETE /identity/v3/projects/0055fb3d13b545e4b33b8ab3cee9053b => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:43.285812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=abe5d40f5ecf4551acb03bf43c4b2294, user_id=8a5ba8f0eddf4df89d8110f6a209602b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:43.287101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.289023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.293551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285ea8c8-24dc-4ae7-9908-1d92c2d57f07 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:43.318413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864749f0-15ae-4486-a053-c21ef2272a9d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.349181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 970/1928] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:43 2022] PUT /identity/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294/users/8a5ba8f0eddf4df89d8110f6a209602b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:43.369299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.372717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.375048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.473520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] MFA Rules not processed for user `8a5ba8f0eddf4df89d8110f6a209602b`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:43.475912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.533405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:43.535830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:43.544716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:43.544716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:43.545035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] PATH_INFO: `/v3/users/7b3468a537ea4330b0bb09839c90ff71` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:43.546172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b3468a537ea4330b0bb09839c90ff71)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:43.548409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:43.671560 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 971/1929] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:43 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:43.908852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:43.908852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:43.950838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b3468a537ea4330b0bb09839c90ff71'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.952021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b3468a537ea4330b0bb09839c90ff71'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.964316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b3468a537ea4330b0bb09839c90ff71'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.995055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b3468a537ea4330b0bb09839c90ff71'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:43.995665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc03dc-9277-4ba8-b457-f705e7c9935c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:43.996627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b3468a537ea4330b0bb09839c90ff71'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.000871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.001986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.002295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.002770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.004878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.023233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.031858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.044118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 972/1930] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:43 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:44.045109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.047152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.077117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9bfee7-fd54-4e6a-8fa3-a16b58791981 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.082698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.083354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.083831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.084114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] PATH_INFO: `/v3/projects/ab9d5ae6b1674054ae040aeab75d31fd` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.085367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab9d5ae6b1674054ae040aeab75d31fd)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.087474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.131111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 959/1931] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:43 2022] DELETE /identity/v3/users/7b3468a537ea4330b0bb09839c90ff71 => generated 0 bytes in 845 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:44.291856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab9d5ae6b1674054ae040aeab75d31fd', 'request_id': 'req-80ec9872-65ef-4a80-ab04-b16fce56e8e0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.307846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab9d5ae6b1674054ae040aeab75d31fd', 'request_id': 'req-80ec9872-65ef-4a80-ab04-b16fce56e8e0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.322493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab9d5ae6b1674054ae040aeab75d31fd', 'request_id': 'req-80ec9872-65ef-4a80-ab04-b16fce56e8e0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.416488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.430662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] The token cache is being invalidate because project ab9d5ae6b1674054ae040aeab75d31fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:44.430662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.432952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.435791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 973/1932] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:44 2022] DELETE /identity/v3/projects/ab9d5ae6b1674054ae040aeab75d31fd => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:44.444181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.565096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4988561d-512f-4345-b1a6-80f0a6442e98 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.660461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.663018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.663018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.663018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] PATH_INFO: `/v3/users/87f5294814d54b21abaf3abca54ecd6f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.664071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87f5294814d54b21abaf3abca54ecd6f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.684681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.696489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:44.769419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:44.770971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:44.798891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ec9872-65ef-4a80-ab04-b16fce56e8e0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:44.809631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:44.810579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:44.812934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:44.814365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] PATH_INFO: `/v3/projects/f836c35943b34e70ab4ff865b3255fd7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:44.817905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f836c35943b34e70ab4ff865b3255fd7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:44.823165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:44.984458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87f5294814d54b21abaf3abca54ecd6f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.985108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87f5294814d54b21abaf3abca54ecd6f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:44.992640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87f5294814d54b21abaf3abca54ecd6f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.010222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87f5294814d54b21abaf3abca54ecd6f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.010913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87f5294814d54b21abaf3abca54ecd6f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.025719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f836c35943b34e70ab4ff865b3255fd7', 'request_id': 'req-781106c0-345d-4c67-b8ea-0713ff51d354'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.033735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f836c35943b34e70ab4ff865b3255fd7', 'request_id': 'req-781106c0-345d-4c67-b8ea-0713ff51d354'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.039532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f836c35943b34e70ab4ff865b3255fd7', 'request_id': 'req-781106c0-345d-4c67-b8ea-0713ff51d354'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.133227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 960/1933] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:44 2022] DELETE /identity/v3/users/87f5294814d54b21abaf3abca54ecd6f => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:45.133227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.133227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.235059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd042512-02ae-41d5-9216-80b661325f17 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.244208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.247507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.247507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.247507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] PATH_INFO: `/v3/users/cd5e059cd4054224ab179ac0e6253d1d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.247507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd5e059cd4054224ab179ac0e6253d1d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.250230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] The token cache is being invalidate because project f836c35943b34e70ab4ff865b3255fd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:45.250498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.250957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.252092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-781106c0-345d-4c67-b8ea-0713ff51d354 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.272934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 974/1934] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:44 2022] DELETE /identity/v3/projects/f836c35943b34e70ab4ff865b3255fd7 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:45.276777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b66f99-0af0-404f-a218-44c2ce2c674a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.279346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b66f99-0af0-404f-a218-44c2ce2c674a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.283060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b66f99-0af0-404f-a218-44c2ce2c674a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.315840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a8b66f99-0af0-404f-a218-44c2ce2c674a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:45.318309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a8b66f99-0af0-404f-a218-44c2ce2c674a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.343909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 975/1935] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:45.347615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.348068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.348558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.381538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:45.383782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.398034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 976/1936] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:45.403986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.416325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cb761d-de43-41ba-b965-ea96074e2056 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.428732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.428732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.428732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.428732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.429480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:45.429480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.431078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.456619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 977/1937] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:45.463897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.486374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecde41be-c5ec-47a6-872f-680d907197fe admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:45.496397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:45.497445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.497913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.499673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.499881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:45.506676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:45.547267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd5e059cd4054224ab179ac0e6253d1d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.549047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd5e059cd4054224ab179ac0e6253d1d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.557619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd5e059cd4054224ab179ac0e6253d1d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.571823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd5e059cd4054224ab179ac0e6253d1d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.572668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-5501f1e2-c51e-4c4d-95c3-bade3f9adcf4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd5e059cd4054224ab179ac0e6253d1d'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:45.748268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 978/1938] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/projects => generated 375 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:45.756888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:45.758728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.803858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 961/1939] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:45 2022] DELETE /identity/v3/users/cd5e059cd4054224ab179ac0e6253d1d => generated 0 bytes in 672 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:45.810998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:45.811897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:45.812443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:45.867598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:45.871007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:45.895408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec71ac8-93e4-47c2-a2d7-784a4cf19c73 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.032204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 962/1940] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:46.044355 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.045707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.165404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.166739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.167430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.168169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.169617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.174080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.222732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91732cd-60f8-4f4f-a988-762017988f8d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.227595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.229224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.229819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.230960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.244107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.245738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.261777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 963/1941] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:46.317433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 979/1942] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:45 2022] POST /identity/v3/users => generated 288 bytes in 558 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:46.326566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.334055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.336740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.353951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307ec63-4c90-4495-ab77-b2a42286e478 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.363887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.365286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.365573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.365844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.367383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.369990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.382470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff865b-699e-4624-bd6f-f96172a35d20 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.386855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 980/1943] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:46.391859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.398826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.399945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.400490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.401507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] PATH_INFO: `/v3/projects/72058380eb344ac2aa055eb04f4e3e48` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.403327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72058380eb344ac2aa055eb04f4e3e48)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.406832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.417386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe6aec7-f5ef-4f01-b12f-49b9fa049a2e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.425469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.426468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.426468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.426757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.427380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.429255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.452726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 981/1944] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:46.457296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.486876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c7cde4-3d99-4c37-b319-ee5065415d34 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.492495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.493307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.493628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.493996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] PATH_INFO: `/v3/projects/26619c5e92de440eb5e812fcca3e37c2/users/f4eb8fd0e5504d4cbeaca0d24d2dd7f4/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.512728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] RBAC: Authorizing `identity:create_grant(project_id=26619c5e92de440eb5e812fcca3e37c2, user_id=f4eb8fd0e5504d4cbeaca0d24d2dd7f4, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.516392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7966d5b-99a9-4276-a950-3d1f9f45b91c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.653192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:46.660243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 982/1945] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:46 2022] PUT /identity/v3/projects/26619c5e92de440eb5e812fcca3e37c2/users/f4eb8fd0e5504d4cbeaca0d24d2dd7f4/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:46.677443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72058380eb344ac2aa055eb04f4e3e48', 'request_id': 'req-2bd51aa1-6ea3-4569-947d-d470e75b1337'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.678892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.679064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.679344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.686617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72058380eb344ac2aa055eb04f4e3e48', 'request_id': 'req-2bd51aa1-6ea3-4569-947d-d470e75b1337'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.695862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72058380eb344ac2aa055eb04f4e3e48', 'request_id': 'req-2bd51aa1-6ea3-4569-947d-d470e75b1337'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.719315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] MFA Rules not processed for user `f4eb8fd0e5504d4cbeaca0d24d2dd7f4`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:46.720838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.843550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 983/1946] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:46.843550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.843550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.848654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] The token cache is being invalidate because project 72058380eb344ac2aa055eb04f4e3e48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:46.849150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.850271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:46.854819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 964/1947] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:46 2022] DELETE /identity/v3/projects/72058380eb344ac2aa055eb04f4e3e48 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:46.862209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:46.865041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:46.951674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d355a28-a2f2-4710-8ba5-cf6aae334e3b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:46.957485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:46.958726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:46.959031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:46.959314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] PATH_INFO: `/v3/users/3c6313fe732b464ba776ffdd9499954d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:46.961231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c6313fe732b464ba776ffdd9499954d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:46.963335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:46.998628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd51aa1-6ea3-4569-947d-d470e75b1337 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.003011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.003643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.004129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.004511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.006393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.027914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.029414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6f9724-651c-47d9-8a9e-ba50d1b63365 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.042168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 965/1948] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:47.043887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da5ef7-333f-4a20-97dc-7493ffeaf644 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.044318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da5ef7-333f-4a20-97dc-7493ffeaf644 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.044629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3da5ef7-333f-4a20-97dc-7493ffeaf644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.112889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f3da5ef7-333f-4a20-97dc-7493ffeaf644 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:47.114298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f3da5ef7-333f-4a20-97dc-7493ffeaf644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.146308 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 966/1949] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:47.168664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.168664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.168664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.198223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:47.201710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.217770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 967/1950] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:47.223019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.235011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858dd395-d8ae-4c74-81f2-bdeef60842c3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.244132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.245606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.246574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.247039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.248273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:47.249293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.252515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.275844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 968/1951] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:02:47.284271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.301439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d6e091-ce06-42a1-b936-ca184179ac83 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.311297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.312407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.312967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.313478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.314880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.318596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.323665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c6313fe732b464ba776ffdd9499954d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.325558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c6313fe732b464ba776ffdd9499954d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.337931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c6313fe732b464ba776ffdd9499954d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.338721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c6313fe732b464ba776ffdd9499954d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.356303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c6313fe732b464ba776ffdd9499954d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:47.564931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 969/1952] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/projects => generated 383 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:47.566358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.568106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.604381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 984/1953] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:46 2022] DELETE /identity/v3/users/3c6313fe732b464ba776ffdd9499954d => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:47.605833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.621616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.698580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f58db62-b3a9-4ded-b9bc-963a7545d7c7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.773492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27626f95-1ee4-47ec-9119-c94622fd4f27 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.784915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.786538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.787754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.789437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.792887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.795548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:47.796236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.796642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.796822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed58ae4-a3a5-40e9-a479-3fb35f9bc5f9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.796920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] PATH_INFO: `/v3/projects/cd5c3596bc744bf49025eed370970c42` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.798184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd5c3596bc744bf49025eed370970c42)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:47.800126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:47.918174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 985/1954] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/users => generated 292 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:47.923134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:47.923395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:47.923873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:47.948432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:47.949670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:47.958372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 986/1955] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:47.959426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:47.981507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db97c546-4e74-4827-b692-ad17b833f059 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:47.998852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd5c3596bc744bf49025eed370970c42', 'request_id': 'req-a3edaa07-9252-442e-b35e-24cd816a4c93'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.008254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd5c3596bc744bf49025eed370970c42', 'request_id': 'req-a3edaa07-9252-442e-b35e-24cd816a4c93'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.028264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd5c3596bc744bf49025eed370970c42', 'request_id': 'req-a3edaa07-9252-442e-b35e-24cd816a4c93'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.170847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.172661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.182941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.189600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] The token cache is being invalidate because project cd5c3596bc744bf49025eed370970c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:48.189990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.191486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.192329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.195452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.197027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.201558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 970/1956] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:47 2022] DELETE /identity/v3/projects/cd5c3596bc744bf49025eed370970c42 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:48.208262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.210159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.230527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 987/1957] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:48.233540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.236550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.256945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d038a6-a153-484c-be1b-553cc0f60ed0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.267074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.268759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.269264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.274129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] PATH_INFO: `/v3/users/8a5ba8f0eddf4df89d8110f6a209602b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.274129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a5ba8f0eddf4df89d8110f6a209602b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.278482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.300739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3edaa07-9252-442e-b35e-24cd816a4c93 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.306215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.308042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.309360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.309731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.312839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.329455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.331172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.358399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 971/1958] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:48.360896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.364448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.390602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56df0a4-b6a4-4476-9116-e1eccfa54460 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.393863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:48.404302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.405348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.405851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.406344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.407495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.410824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.437879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 972/1959] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:48.444326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.455707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab7891-9eb9-434b-ad45-0fca44b5a9bd admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.460745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.461644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.461883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.462176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] PATH_INFO: `/v3/projects/6dc95c98229e4c73b8f037a79c75199e/users/90dce34f6ca5467bbc8e51bc8f074796/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.486438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dc95c98229e4c73b8f037a79c75199e, user_id=90dce34f6ca5467bbc8e51bc8f074796, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:48.489630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4de5de-5fd0-4c7c-a2a7-20864d41077e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:48.604648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a5ba8f0eddf4df89d8110f6a209602b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.606002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a5ba8f0eddf4df89d8110f6a209602b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.621338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a5ba8f0eddf4df89d8110f6a209602b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.622507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a5ba8f0eddf4df89d8110f6a209602b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.646542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a5ba8f0eddf4df89d8110f6a209602b'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:48.705975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 973/1960] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:48 2022] PUT /identity/v3/projects/6dc95c98229e4c73b8f037a79c75199e/users/90dce34f6ca5467bbc8e51bc8f074796/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:48.717419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.717730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.717975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:48.756465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] MFA Rules not processed for user `90dce34f6ca5467bbc8e51bc8f074796`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:48.762574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.772267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 988/1961] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:48 2022] DELETE /identity/v3/users/8a5ba8f0eddf4df89d8110f6a209602b => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:48.860801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 974/1962] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:48.876564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:48.879295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:48.950621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fbb55b-e0d0-46e5-983b-736fcc643b13 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:48.954783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:48.972831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:48.972831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:48.972831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:49.033119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:49.034683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96efedaa-b0e5-46b9-a4dc-8f52b07de16c service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:49.047799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 989/1963] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:49.609602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:49.611724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:49.765723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363403f3-7469-458c-8cc4-be478240cb7d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:49.873083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:49.874600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:49.875043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:49.875462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] PATH_INFO: `/v3/projects/510c702dfdf6457aabafadfc3325ff2f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:49.876831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510c702dfdf6457aabafadfc3325ff2f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:49.878957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:49.906511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979cfd-8b7e-43ee-ab52-87a15973d57f None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:49.906880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979cfd-8b7e-43ee-ab52-87a15973d57f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:49.907386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87979cfd-8b7e-43ee-ab52-87a15973d57f None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:49.909440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 990/1964] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:02:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:02:49.921299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:49.923424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:49.923978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:49.958551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:49.960785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:50.086297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 991/1965] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:02:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:50.102710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:50.117752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:50.119820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:50.141801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a7bc97-2d84-454c-b4af-402be7d38efd None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:50.150590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:50.151556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:50.152194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:50.152615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:50.166037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:50.169341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:50.195385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 992/1966] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:50.229768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510c702dfdf6457aabafadfc3325ff2f', 'request_id': 'req-65f94fd0-de54-4753-8a1e-4f58f505b510'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.276956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510c702dfdf6457aabafadfc3325ff2f', 'request_id': 'req-65f94fd0-de54-4753-8a1e-4f58f505b510'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.292418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510c702dfdf6457aabafadfc3325ff2f', 'request_id': 'req-65f94fd0-de54-4753-8a1e-4f58f505b510'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.455471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] The token cache is being invalidate because project 510c702dfdf6457aabafadfc3325ff2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:50.455471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.458601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.464402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 975/1967] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:49 2022] DELETE /identity/v3/projects/510c702dfdf6457aabafadfc3325ff2f => generated 0 bytes in 854 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:50.465347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:50.476787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56438d9d-8cf4-4eca-92da-a6234e3d79e4 service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:50.543010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:50.543666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:50.543949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:50.544257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] PATH_INFO: `/v3/users/24b4dd5c94464efdbbb71197324d9153` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:50.545520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] RBAC: Authorizing `identity:delete_user(user_id=24b4dd5c94464efdbbb71197324d9153)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:50.557528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:50.568624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:50.570289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:50.603391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f94fd0-de54-4753-8a1e-4f58f505b510 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:50.639645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:50.641956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:50.643041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:50.643747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] PATH_INFO: `/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:50.656202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=abe5d40f5ecf4551acb03bf43c4b2294)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:50.661066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:50.978891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24b4dd5c94464efdbbb71197324d9153'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:50.984210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24b4dd5c94464efdbbb71197324d9153'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.007365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abe5d40f5ecf4551acb03bf43c4b2294', 'request_id': 'req-50003437-e3c4-4a17-a811-abf141ac6eb2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.022309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24b4dd5c94464efdbbb71197324d9153'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.025081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24b4dd5c94464efdbbb71197324d9153'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.049375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abe5d40f5ecf4551acb03bf43c4b2294', 'request_id': 'req-50003437-e3c4-4a17-a811-abf141ac6eb2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.097697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abe5d40f5ecf4551acb03bf43c4b2294', 'request_id': 'req-50003437-e3c4-4a17-a811-abf141ac6eb2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.137077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-202b534d-2079-4977-8c07-818b2c215bca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24b4dd5c94464efdbbb71197324d9153'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.400210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] The token cache is being invalidate because project abe5d40f5ecf4551acb03bf43c4b2294 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:51.400723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.401938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-50003437-e3c4-4a17-a811-abf141ac6eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:51.408267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 976/1968] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:50 2022] DELETE /identity/v3/projects/abe5d40f5ecf4551acb03bf43c4b2294 => generated 0 bytes in 837 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:51.412695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073598a-2df3-428f-a677-d887da769c5b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:51.413163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073598a-2df3-428f-a677-d887da769c5b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:51.413626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073598a-2df3-428f-a677-d887da769c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:51.460592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6073598a-2df3-428f-a677-d887da769c5b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:51.462808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6073598a-2df3-428f-a677-d887da769c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:51.496828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 993/1969] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:02:50 2022] DELETE /identity/v3/users/24b4dd5c94464efdbbb71197324d9153 => generated 0 bytes in 1032 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:51.563542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 977/1970] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:51.580596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:51.581441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:51.582048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:51.595301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:51.638861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:51.642514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:51.781612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 994/1971] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:51.788021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073598a-2df3-428f-a677-d887da769c5b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:51.822129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073598a-2df3-428f-a677-d887da769c5b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:51.833132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:51.834803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:51.834803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:51.835604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:51.837423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:51.838253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:51.841771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:51.887296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 978/1972] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:51.890178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:51.902438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c241814-6ab4-4ba7-b0de-d45ecd8e255c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:51.907626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:51.908473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:51.908640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:51.908967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:51.909707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:51.911775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.075524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 995/1973] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:51 2022] POST /identity/v3/projects => generated 375 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:52.083552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.117303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17257ba3-1d4a-4fac-832f-ce060edcb471 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.171989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.175092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:52.240036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.240693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.240974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.241249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.242270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.251883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.309426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2805f9-823e-472e-a3a8-4ed1cd703f11 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.317194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.318117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.318551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.318995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] PATH_INFO: `/v3/projects/d7ed1848eff24316acf6d6bba3f3347b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.320580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7ed1848eff24316acf6d6bba3f3347b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.323157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.401035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 979/1974] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/users => generated 288 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:52.404417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.412182 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:52.424466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bd75ea-4d57-4511-a1d6-2753ff2beba2 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.436851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.438409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.440311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.440808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.441887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.444696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.461349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 980/1975] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:02:52.465533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.488338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad50ffaa-04cb-494e-8ed5-6eccfe539bca admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.520622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7ed1848eff24316acf6d6bba3f3347b', 'request_id': 'req-9b596058-42ee-4431-a128-bf11382af78e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.531353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7ed1848eff24316acf6d6bba3f3347b', 'request_id': 'req-9b596058-42ee-4431-a128-bf11382af78e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.538212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7ed1848eff24316acf6d6bba3f3347b', 'request_id': 'req-9b596058-42ee-4431-a128-bf11382af78e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.634152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.634814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.635097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.635459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.636202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.638435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.651278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 981/1976] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:52.670075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.682497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8637cff7-7d96-4def-94be-fe7835c15ed7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:52.688657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:52.689744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.690124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.690457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] PATH_INFO: `/v3/projects/acfb95aea5774704bf6bc2312974c90e/users/197645b7201a4d05a810e6dc0cceebd0/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.707903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acfb95aea5774704bf6bc2312974c90e, user_id=197645b7201a4d05a810e6dc0cceebd0, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:52.710956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace48b10-12c1-4cb3-9f29-0ea4eb1d2e66 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:52.776078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] The token cache is being invalidate because project d7ed1848eff24316acf6d6bba3f3347b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:02:52.776078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.776078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9b596058-42ee-4431-a128-bf11382af78e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:02:52.776078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 996/1977] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:02:52 2022] DELETE /identity/v3/projects/d7ed1848eff24316acf6d6bba3f3347b => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:52.776078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ff552-a22b-4ce9-b866-ecdac9bffe09 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.776930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ff552-a22b-4ce9-b866-ecdac9bffe09 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.776930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866ff552-a22b-4ce9-b866-ecdac9bffe09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.801679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-866ff552-a22b-4ce9-b866-ecdac9bffe09 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:52.802965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-866ff552-a22b-4ce9-b866-ecdac9bffe09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:52.815104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 997/1978] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:52.823598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.824018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.824427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.828499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 982/1979] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:52 2022] PUT /identity/v3/projects/acfb95aea5774704bf6bc2312974c90e/users/197645b7201a4d05a810e6dc0cceebd0/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:52.830880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:52.831173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:52.831449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:52.849654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:52.851202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:52.868885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] MFA Rules not processed for user `197645b7201a4d05a810e6dc0cceebd0`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:52.870276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:52.974616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 983/1980] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:52.980403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:52.982890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.017325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 998/1981] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:02:53.022069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.033736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb21d4-b96d-469c-83d2-417702cb4d4a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.044537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.044537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.044537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.044537 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.045333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:02:53.045333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.045333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.059249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 999/1982] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:02:53.066139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.081767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a7f529-0203-4d99-8202-6fbab52eccd5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.100664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cafaa0-26fe-4bed-b50b-72722d2e711c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.102198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.103936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.104625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.105199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.106174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.107819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.108587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.108873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.109150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.109960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.111161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.113613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.259447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 984/1983] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:52 2022] POST /identity/v3/projects => generated 375 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:02:53.260549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1000/1984] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:02:53 2022] POST /identity/v3/projects => generated 377 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:53.268338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.268571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.359830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b80db0c-fa66-44e8-b874-8bc952509a9f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.369896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.369896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.369896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.369896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.370397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.370510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eaed08-b6e2-4070-964f-e3c355d9fe94 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.370673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.376088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.376943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.377441 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.378748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.379023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.382131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.483466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1001/1985] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:53 2022] POST /identity/v3/users => generated 294 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:53.488012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.501272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748dadff-e85a-45ed-bab0-d731dd3c4025 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.506602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.507227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.507510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.507784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.508468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.510087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.526437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1002/1986] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:53.526437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.527479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 985/1987] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:02:53 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:02:53.528844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.537144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d2e478-39fa-4278-8397-4126d86beb26 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.540037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2e232-3067-4bb9-bbcd-1c90fc83b844 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.542195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.542943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.543233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.543511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.544571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.545940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.546255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.547143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.547143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.548028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] PATH_INFO: `/v3/projects/56b814e49cbc45029a65fea6aa00434b/users/d22b958f06ec4543960e1a1a175306ef/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.558907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1003/1988] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:53.567496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.577332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b814e49cbc45029a65fea6aa00434b, user_id=d22b958f06ec4543960e1a1a175306ef, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.578735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdac4166-62f9-40f9-9aa5-bd275f93012c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.580292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.586360 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.586977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.587255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.587526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.588208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.589924 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.601846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1004/1989] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:53.606900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.618484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4010dd-b039-4163-b67d-e26d5ac33b14 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.624480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.625162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.625628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.625867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] PATH_INFO: `/v3/projects/550a0d254a4f4d6c9d5645bf2a9fb0fa/users/fce05334b66946f49fefc4e8b36f2c9c/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.656494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=550a0d254a4f4d6c9d5645bf2a9fb0fa, user_id=fce05334b66946f49fefc4e8b36f2c9c, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.659663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e485cd1c-46e4-4674-96ea-96b8d3447aeb admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.694075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 986/1990] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:53 2022] PUT /identity/v3/projects/56b814e49cbc45029a65fea6aa00434b/users/d22b958f06ec4543960e1a1a175306ef/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:02:53.699630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.799379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1005/1991] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:53 2022] PUT /identity/v3/projects/550a0d254a4f4d6c9d5645bf2a9fb0fa/users/fce05334b66946f49fefc4e8b36f2c9c/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:53.806054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae9dbb3-3bd5-4f36-a9cb-77c0018b3763 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.810711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.811108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.811390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.827320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:53.848721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] MFA Rules not processed for user `fce05334b66946f49fefc4e8b36f2c9c`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:53.850134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.894261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.895567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.895845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.896293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.899373 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.901200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.924419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1006/1992] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:53.932236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 987/1993] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:02:53.937027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.943846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:53.943846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:53.951145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc8cbc1-3e3f-49cc-b4d9-214ccff8d001 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:53.958585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:53.959240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:53.959591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:53.959591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] PATH_INFO: `/v3/projects/56b814e49cbc45029a65fea6aa00434b/users/d22b958f06ec4543960e1a1a175306ef/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:53.963399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56b814e49cbc45029a65fea6aa00434b, user_id=d22b958f06ec4543960e1a1a175306ef, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:53.966942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b028b0c0-1d66-4fbd-8ccb-91c0974f5991 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:53.993017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1007/1994] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:02:53 2022] PUT /identity/v3/projects/56b814e49cbc45029a65fea6aa00434b/users/d22b958f06ec4543960e1a1a175306ef/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:02:54.007534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.007859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.008733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.041352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] MFA Rules not processed for user `d22b958f06ec4543960e1a1a175306ef`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:02:54.042612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.061542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0da7d7c-7e4c-4381-9983-b0b58b749bdc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.116707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.117685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.118187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.118707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.132212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1008/1995] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:02:54 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:02:54.141733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:02:54.145078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:02:54.231583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.233179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.234826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7f9f0b-b784-4272-b7a4-13f86843e316 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:02:54.238809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:02:54.242223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:02:54.242223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:02:54.242223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:02:54.242816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 988/1996] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:53 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:02:54.328370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:02:54.330500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf57e322-3126-452f-9e01-529e583b6f51 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:02:54.345331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1009/1997] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:02:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:02:54.986357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:55.445282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:56.999006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:57.190189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:58.024114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:58.223152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:58.910304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:02:59.737418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:00.553351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:01.244770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:01.736674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:02.184429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:02.850463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:04.010622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:04.367280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:04.407860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:04.499583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a20352e-8019-4b66-95e4-c828b644ed9e service glance] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:04.506918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:04.508496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:04.508496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:04.508496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] PATH_INFO: `/v3/users/197645b7201a4d05a810e6dc0cceebd0` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:04.509683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] RBAC: Authorizing `identity:delete_user(user_id=197645b7201a4d05a810e6dc0cceebd0)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:04.512240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:04.788725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '197645b7201a4d05a810e6dc0cceebd0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:04.789394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '197645b7201a4d05a810e6dc0cceebd0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:04.797342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '197645b7201a4d05a810e6dc0cceebd0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:04.821642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '197645b7201a4d05a810e6dc0cceebd0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:04.822438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '197645b7201a4d05a810e6dc0cceebd0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:04.927374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 989/1998] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:04 2022] DELETE /identity/v3/users/197645b7201a4d05a810e6dc0cceebd0 => generated 0 bytes in 520 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:04.932675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:04.932985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:04.933457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:04.954352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:04.955482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:05.017802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1010/1999] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:05.028067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:05.029807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:05.107861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b8dba5-c45d-46a8-9b6b-d2409a21fb07 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:05.111802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:05.112460 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:05.112884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:05.113179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:05.124402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:05.125890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:05.135930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 990/2000] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:05.640047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:05.642026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:05.654632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90bbab2-a86c-434e-a13b-cef1b9ee434f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:05.659428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:05.660063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:05.660375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:05.660653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] PATH_INFO: `/v3/users/90dce34f6ca5467bbc8e51bc8f074796` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:05.661818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90dce34f6ca5467bbc8e51bc8f074796)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:05.663838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:05.681240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:05.683234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:05.703509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007c67de-df1a-42d9-adfe-212077f4dcbd service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:05.711835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:05.712626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:05.713175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:05.713693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] PATH_INFO: `/v3/projects/acfb95aea5774704bf6bc2312974c90e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:05.716545 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] RBAC: Authorizing `identity:delete_project(project_id=acfb95aea5774704bf6bc2312974c90e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:05.719134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:05.778942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:05.968087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90dce34f6ca5467bbc8e51bc8f074796'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:05.969070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90dce34f6ca5467bbc8e51bc8f074796'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:05.974959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acfb95aea5774704bf6bc2312974c90e', 'request_id': 'req-588e55f0-9898-49cf-9e39-5eb9f7a65082'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:05.979625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90dce34f6ca5467bbc8e51bc8f074796'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:05.985545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90dce34f6ca5467bbc8e51bc8f074796'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:05.987339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acfb95aea5774704bf6bc2312974c90e', 'request_id': 'req-588e55f0-9898-49cf-9e39-5eb9f7a65082'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.000389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acfb95aea5774704bf6bc2312974c90e', 'request_id': 'req-588e55f0-9898-49cf-9e39-5eb9f7a65082'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.009042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-12d5a090-7383-4060-9bbd-75b89f709dfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90dce34f6ca5467bbc8e51bc8f074796'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.107172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] The token cache is being invalidate because project acfb95aea5774704bf6bc2312974c90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:06.107805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.108128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.123818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 991/2001] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:05 2022] DELETE /identity/v3/projects/acfb95aea5774704bf6bc2312974c90e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:06.123818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:06.127769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:06.155360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:06.184440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1011/2002] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:05 2022] DELETE /identity/v3/users/90dce34f6ca5467bbc8e51bc8f074796 => generated 0 bytes in 545 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:06.191340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:06.191720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:06.192074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:06.214010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:06.215847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:06.221587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588e55f0-9898-49cf-9e39-5eb9f7a65082 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:06.293009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:06.294082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:06.294557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:06.294847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] PATH_INFO: `/v3/users/d22b958f06ec4543960e1a1a175306ef` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:06.296327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22b958f06ec4543960e1a1a175306ef)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:06.298417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:06.298769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1012/2003] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:06.314765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:06.316279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:06.392667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb08c2d-bc96-4d0e-919a-f0bf4f3c324c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:06.396510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:06.406453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:06.406453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:06.406453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:06.409122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:06.411092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:06.423839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1013/2004] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:06.572028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22b958f06ec4543960e1a1a175306ef'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.572947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22b958f06ec4543960e1a1a175306ef'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.582217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22b958f06ec4543960e1a1a175306ef'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.596017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22b958f06ec4543960e1a1a175306ef'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.596797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22b958f06ec4543960e1a1a175306ef'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:06.682559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 992/2005] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:06 2022] DELETE /identity/v3/users/d22b958f06ec4543960e1a1a175306ef => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:06.899005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:06.903284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:06.939704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a0d16-7cf0-4235-abfa-5bdf6ea64a66 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:06.993827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:07.064869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:07.065707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.066064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.067158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] PATH_INFO: `/v3/projects/6dc95c98229e4c73b8f037a79c75199e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.068501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dc95c98229e4c73b8f037a79c75199e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:07.070541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:07.289320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dc95c98229e4c73b8f037a79c75199e', 'request_id': 'req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.296933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dc95c98229e4c73b8f037a79c75199e', 'request_id': 'req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.302753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dc95c98229e4c73b8f037a79c75199e', 'request_id': 'req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.326110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:07.328425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:07.346272 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebf4121-4892-4c6d-9345-b84ebe9ff8f3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:07.443505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] The token cache is being invalidate because project 6dc95c98229e4c73b8f037a79c75199e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:07.447687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.447687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c137e2dc-c6c9-4725-946f-ca0e7ba65fee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.449425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1014/2006] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:06 2022] DELETE /identity/v3/projects/6dc95c98229e4c73b8f037a79c75199e => generated 0 bytes in 553 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:07.454427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba7fd-f8b2-417e-a30b-b9cbf74e5683 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.454779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba7fd-f8b2-417e-a30b-b9cbf74e5683 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.455282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2ba7fd-f8b2-417e-a30b-b9cbf74e5683 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.483014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dc2ba7fd-f8b2-417e-a30b-b9cbf74e5683 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:07.487964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc2ba7fd-f8b2-417e-a30b-b9cbf74e5683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:07.508351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:07.508945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.509390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.510129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] PATH_INFO: `/v3/projects/56b814e49cbc45029a65fea6aa00434b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.512181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56b814e49cbc45029a65fea6aa00434b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:07.512394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:07.514731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1015/2007] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:07.515124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:07.533743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.534145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.534534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.558686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:07.559997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:07.573447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1016/2008] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:07.577681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:07.587296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f1c03e-23c8-4411-b500-86b6ab8579f7 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:07.592873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:07.593483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.593769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.594047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.594697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:07.595004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:07.596645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:07.609745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1017/2009] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:07.614662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:07.625881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c01d4d-8955-489f-83e1-6e87d2a0cb9c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:07.632014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:07.633039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.633039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.633505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.634198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:07.635862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:07.736717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56b814e49cbc45029a65fea6aa00434b', 'request_id': 'req-58c858e3-fc16-4ad5-90fb-01d58f634b43'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.744251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56b814e49cbc45029a65fea6aa00434b', 'request_id': 'req-58c858e3-fc16-4ad5-90fb-01d58f634b43'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.750798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56b814e49cbc45029a65fea6aa00434b', 'request_id': 'req-58c858e3-fc16-4ad5-90fb-01d58f634b43'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.826324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1018/2010] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:07 2022] POST /identity/v3/projects => generated 375 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:07.829950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:07.854514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:07.916828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de87da8c-30d1-4532-b45d-42fda04f7014 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:07.922487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:07.923355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.923659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.924010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.924751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:07.927065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:07.938935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] The token cache is being invalidate because project 56b814e49cbc45029a65fea6aa00434b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:07.939155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.943933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-58c858e3-fc16-4ad5-90fb-01d58f634b43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:07.943933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 993/2011] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:07 2022] DELETE /identity/v3/projects/56b814e49cbc45029a65fea6aa00434b => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:07.947419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8da52-549b-4c93-a0cb-208740662cd5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:07.947705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8da52-549b-4c93-a0cb-208740662cd5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:07.948007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee8da52-549b-4c93-a0cb-208740662cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:07.973236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8ee8da52-549b-4c93-a0cb-208740662cd5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:07.975347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8ee8da52-549b-4c93-a0cb-208740662cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:07.987359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 994/2012] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:08.002456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.002738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.003058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.023296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:08.024325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:08.032873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 995/2013] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:08.037462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.047881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57599a5b-2abf-4369-a89f-da5535a05747 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.055587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.056303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.056644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.056944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.058158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:08.058514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.061754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.091968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 996/2014] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:08.093018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1019/2015] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:07 2022] POST /identity/v3/users => generated 288 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:08.095967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.098483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:08.100081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.118585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bc8c16-f823-473c-a3ff-d762b3d41b59 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.123967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e695ea-60cc-46cd-adb2-457ea7df60c4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.137834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.137834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.137834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.137834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.138489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.138656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.138656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.138656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.138656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.139206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.139358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.139358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.161042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 997/2016] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:08.162668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.177426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b7bceb-4be3-4f3e-ab34-49b1dd239e63 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.182404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.183019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.183352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.184234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.184887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.186498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.199564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 998/2017] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:08.204499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.216431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc5d95d-c993-4ac1-8f96-de493adccf6f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.221256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.221926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.222203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.222475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] PATH_INFO: `/v3/projects/4017be3f15894016a44bc4293f890268/users/5e9d4d3d07cd4ef08c791381d118f883/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.240354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4017be3f15894016a44bc4293f890268, user_id=5e9d4d3d07cd4ef08c791381d118f883, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.247730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b83ca80-bf49-43b6-a317-13a270fce75b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.336523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1020/2018] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/projects => generated 375 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:08.343134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.392401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 999/2019] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:08 2022] PUT /identity/v3/projects/4017be3f15894016a44bc4293f890268/users/5e9d4d3d07cd4ef08c791381d118f883/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:08.406154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.406490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.406695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.420588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4caccb-d60b-497a-b41d-f1a99ff2cd23 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.463395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] MFA Rules not processed for user `5e9d4d3d07cd4ef08c791381d118f883`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:08.464883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:08.520361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.521049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.521334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.521672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.522313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.524608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.561360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1000/2020] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:08.562461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.577790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35814d7b-0bd2-4b10-86f7-6d1ad09b5adb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.583148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.583850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.590069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.590069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.590069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.590069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.618987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1021/2021] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/users => generated 288 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:08.624642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.631369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46f8a-c9c2-428b-91a0-3b1582aa7d3e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.636220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.636877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.637218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.637541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.638327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.639937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.651773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1022/2022] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:08.656691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.667039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5aabd-30fc-470a-a2aa-26b2427c2035 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.673208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.673834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.675876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.675876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.675876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.676941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.689874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1023/2023] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:08.694139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.706424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f6c1ec-71ba-47cf-8a2c-cb2ce2bdc8a9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.723400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1001/2024] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/projects => generated 375 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:08.724605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:08.794294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.795007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.795289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.795596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] PATH_INFO: `/v3/projects/d1c0f75e0b224f3f924919454f5c7e46/users/0f1265a57a60483ebb643f47e456a0b9/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.825257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1c0f75e0b224f3f924919454f5c7e46, user_id=0f1265a57a60483ebb643f47e456a0b9, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.827531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6fe35a-38e6-4252-887b-5d98b5f3d3a4 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.848713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38bfb5cd-46ad-409d-b72e-552ac29b5807 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:08.853586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:08.854209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.854489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.854787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.856005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:08.858023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:08.913260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1024/2025] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:08 2022] PUT /identity/v3/projects/d1c0f75e0b224f3f924919454f5c7e46/users/0f1265a57a60483ebb643f47e456a0b9/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:08.927219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:08.928648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:08.928976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:08.943962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:08.976521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] MFA Rules not processed for user `0f1265a57a60483ebb643f47e456a0b9`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:08.977783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:09.003293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1002/2026] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:09.004708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.015077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780b5b31-1aa4-4485-b371-fc4eb236630f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:09.051393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1025/2027] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:09.065138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.067035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:09.095993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:09.096656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.096936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.097209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.097869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:09.099550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:09.111385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1003/2028] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:09.115606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.125807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91c14e7-52af-4478-b353-22930ac37606 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:09.130603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:09.131219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.131496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.131768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.131914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf59c00-f88e-4c23-a04a-563a41e78ba5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:09.132452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:09.134072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:09.136085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:09.136793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.137087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.137361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.145566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1004/2029] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:09.152823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:09.152823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d624c269-79b0-400f-b044-8b66cebd54d0 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:09.153079 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.160849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dc3c3d-f610-4085-ac3d-afba2d223263 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:09.161094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1026/2030] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:09.165971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:09.166700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.166895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.167173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] PATH_INFO: `/v3/projects/beac19b524ae4d6a848caf5c96ca4709/users/0a2ba434496e484c91fbec46cde4e7c1/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.184652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:09.190516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beac19b524ae4d6a848caf5c96ca4709, user_id=0a2ba434496e484c91fbec46cde4e7c1, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:09.195779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:09.315779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1005/2031] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:09 2022] PUT /identity/v3/projects/beac19b524ae4d6a848caf5c96ca4709/users/0a2ba434496e484c91fbec46cde4e7c1/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:09.323471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.323769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.324039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.365347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] MFA Rules not processed for user `0a2ba434496e484c91fbec46cde4e7c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:09.366796 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:09.463942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1027/2032] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:09.465379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.617525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460aa21-e522-4b92-88d8-7bbdc97568b3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:09.630993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:09.633438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:09.638650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:09.638650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:09.640442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:09.648600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:09.843194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1006/2033] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:09 2022] POST /identity/v3/projects => generated 375 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:09.844640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:09.859872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a0b0a1-4eac-45b2-a8e9-62658a351441 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.026088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.026905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.026905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.028012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.030015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:10.032011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:10.204784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1028/2034] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:09 2022] POST /identity/v3/users => generated 294 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:10.210240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:10.241677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e67b007-2c84-41a4-94fc-3ce3e707c9ae admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.253795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:10.258105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.264928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.264928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.264928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.264928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:10.264928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:10.281779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1007/2035] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:10.287958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:10.313719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207da695-5481-41f6-bdef-a70a1b622d1e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.330208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.331405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.331883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.332408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] PATH_INFO: `/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d/users/abb1c8ef24a74fb5b1f086d5306435c1/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.365930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fdbf8d0cfa49b78c10f9c56958699d, user_id=abb1c8ef24a74fb5b1f086d5306435c1, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:10.370814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:10.465020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1029/2036] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:10 2022] PUT /identity/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d/users/abb1c8ef24a74fb5b1f086d5306435c1/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:10.469440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:10.480935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febda1b1-afad-48db-b5e5-f9e4c1dc1caf admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.553391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.554271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.554669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.555082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.555928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:10.558069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b72ee-c453-4b37-bf2e-e09f20e14574 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:10.572719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1008/2037] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:10 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:10.577398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:10.592885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d4149e-295a-4443-bc84-818bc656f385 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.601094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.601887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.602237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.602556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] PATH_INFO: `/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d/users/abb1c8ef24a74fb5b1f086d5306435c1/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.605114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6fdbf8d0cfa49b78c10f9c56958699d, user_id=abb1c8ef24a74fb5b1f086d5306435c1, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:10.608516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:10.625350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1030/2038] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:10 2022] PUT /identity/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d/users/abb1c8ef24a74fb5b1f086d5306435c1/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:10.638948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.639426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.639893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:10.672988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] MFA Rules not processed for user `abb1c8ef24a74fb5b1f086d5306435c1`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:10.674164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:10.748459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:10.783339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1009/2039] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:10.824518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:10.906315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4da679f-739c-490f-b66c-a44bbb463390 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:10.912454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:10.913504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:10.914010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:10.914511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:11.002671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:11.004188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:11.015483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1031/2040] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:12.396066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:13.007366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:14.133433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:14.199689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:15.044941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:15.156582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:15.583830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:15.586293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:15.602734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7713c43-2e71-457a-be7e-26893e69fd7f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:15.607601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:15.619218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:15.619218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:15.619218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:15.707685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:15.710500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0ecb262-4e03-4204-bfae-0d95605bb30f service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:15.727185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1010/2041] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:16.240919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:16.389973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:16.530311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7849f29c-4652-4c18-b210-67b40c846154 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:16.535364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:16.536009 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:16.536334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:16.536605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] PATH_INFO: `/v3/users/0f1265a57a60483ebb643f47e456a0b9` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:16.537783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f1265a57a60483ebb643f47e456a0b9)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:16.539734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:17.100090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f1265a57a60483ebb643f47e456a0b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:17.100938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f1265a57a60483ebb643f47e456a0b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:17.110764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f1265a57a60483ebb643f47e456a0b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:17.111542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f1265a57a60483ebb643f47e456a0b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:17.135770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f1265a57a60483ebb643f47e456a0b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:17.161764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:17.242146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1032/2042] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:16 2022] DELETE /identity/v3/users/0f1265a57a60483ebb643f47e456a0b9 => generated 0 bytes in 849 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:17.245491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:17.245971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:17.246352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:17.268187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:17.269491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:17.334637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:17.345670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1011/2043] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:17.357101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:17.358607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:17.430110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed4904c-bc5e-47b2-ac49-f8dddbd8af88 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:17.434592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:17.435267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:17.435564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:17.435991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:17.447235 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:17.448768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:17.460321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1033/2044] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:18.027926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:18.031657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:18.076569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4026a028-1562-48ca-9138-4f972cf99588 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:18.083333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:18.084307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.084897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.085268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] PATH_INFO: `/v3/projects/d1c0f75e0b224f3f924919454f5c7e46` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.087436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1c0f75e0b224f3f924919454f5c7e46)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:18.089540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:18.145185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:18.147574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:18.246728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9190971-d77b-4d74-8f3e-fc526cb367bd service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:18.251014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:18.255455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.255455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.255455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.255455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:18.332347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1c0f75e0b224f3f924919454f5c7e46', 'request_id': 'req-ff81cb66-d617-40c1-91a2-cd427c3bb004'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:18.343220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1c0f75e0b224f3f924919454f5c7e46', 'request_id': 'req-ff81cb66-d617-40c1-91a2-cd427c3bb004'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:18.355920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1c0f75e0b224f3f924919454f5c7e46', 'request_id': 'req-ff81cb66-d617-40c1-91a2-cd427c3bb004'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:18.367653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:18.369636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea815f8-3757-46e9-aac6-2dc1f767325e service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:18.385953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1034/2045] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 242 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:18.479409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] The token cache is being invalidate because project d1c0f75e0b224f3f924919454f5c7e46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:18.479711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:18.480972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ff81cb66-d617-40c1-91a2-cd427c3bb004 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:18.482604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1012/2046] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:18 2022] DELETE /identity/v3/projects/d1c0f75e0b224f3f924919454f5c7e46 => generated 0 bytes in 456 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:18.487765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.488077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.488394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.521445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:18.521445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:18.589437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1035/2047] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:18.609207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.609501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.609785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.633356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:18.634647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:18.647199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1013/2048] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:18.653645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:18.666652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef8b8f-25a6-4e27-a222-fc7603eadf6b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:18.672938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:18.674031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.674329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.674879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.675245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:18.675606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:18.677565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:18.694984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1036/2049] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:18.699634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:18.720874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2d3036-96d7-47e7-9ef3-cd606c225880 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:18.732391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693393d5-3069-417a-b323-052305a421df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:18.734623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693393d5-3069-417a-b323-052305a421df admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:18.735255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693393d5-3069-417a-b323-052305a421df admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:18.735924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693393d5-3069-417a-b323-052305a421df admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:18.737224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693393d5-3069-417a-b323-052305a421df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:18.740781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693393d5-3069-417a-b323-052305a421df admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:18.922930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1014/2050] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:18 2022] POST /identity/v3/projects => generated 369 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:18.927758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:18.938784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0b004-312e-4b54-acb6-40bcd799c7cf admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:19.005226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:19.016464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:19.017813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.017813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.017813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.018296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:19.020656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:19.034147 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:19.140887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1037/2051] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:18 2022] POST /identity/v3/users => generated 285 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:19.143254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693393d5-3069-417a-b323-052305a421df admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:19.157958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693393d5-3069-417a-b323-052305a421df admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:19.166783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:19.168390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.168779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.169244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.170365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:19.173274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:19.195451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1015/2052] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:19.196785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:19.208511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0774f11-b38d-4f48-bc10-302aa30ecd5d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:19.213873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:19.217705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.217705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.217705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.217705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:19.217705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701df14f-f992-4ab5-8550-d79cc15be5f9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:19.231638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1038/2053] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:19.236024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:19.250534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffc64f9-f0dc-4ee0-a6d2-c2f83ac1d60d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:19.259818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:19.262523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.263035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.263520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] PATH_INFO: `/v3/projects/9e36b71df75a4f69b93ee08575030e3c/users/b3161900d4434cef94776d7b2e27f261/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.296738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e36b71df75a4f69b93ee08575030e3c, user_id=b3161900d4434cef94776d7b2e27f261, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:19.302950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:19.425709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:19.434325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1016/2054] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:19 2022] PUT /identity/v3/projects/9e36b71df75a4f69b93ee08575030e3c/users/b3161900d4434cef94776d7b2e27f261/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:19.445112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662687b-fe72-42fd-9d90-815f450d1541 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.445797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662687b-fe72-42fd-9d90-815f450d1541 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.445797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662687b-fe72-42fd-9d90-815f450d1541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.479997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e662687b-fe72-42fd-9d90-815f450d1541 None None] MFA Rules not processed for user `b3161900d4434cef94776d7b2e27f261`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:19.481326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e662687b-fe72-42fd-9d90-815f450d1541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:19.593965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1039/2055] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:19.611838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:19.613407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:19.722103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d16cde-58ba-4c5e-ac66-f3d71758724f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:19.726589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:19.728783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:19.729170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:19.729426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:19.744747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:19.746818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99951f8-bf47-4cd4-9b31-96caa1e52536 service glance] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:19.767998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1017/2056] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:19 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:19.843242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:20.178896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:20.321761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:21.950407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:22.059795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:23.277250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:23.605949 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:23.749133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:23.749440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:23.749755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:23.751285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1040/2057] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:03:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:03:23.771219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:23.783767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:23.783767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:23.798111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:23.799450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:23.913993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1018/2058] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:03:23 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:23.944621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:23.946475 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:23.959640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:23.964368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:23.991518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1070c032-88ed-4a82-b9c6-9409e2492f75 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:24.002339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:24.003121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.003388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.003657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.015276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:24.017174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:24.031190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1041/2059] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:24.032681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:24.034504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:24.086584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5316d9a3-3d32-48a3-9d1b-1111f5d78c5c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:24.093209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:24.095286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.095577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.096075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] PATH_INFO: `/v3/users/c96336eb4bfd461e82e7761077a5b910` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.097749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c96336eb4bfd461e82e7761077a5b910)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:24.099701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:24.146607 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00531b10-2f7f-498c-a6f9-e6f5b8aa9370 service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:24.151354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:24.152101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.152348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.152618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] PATH_INFO: `/v3/users/f4eb8fd0e5504d4cbeaca0d24d2dd7f4` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.153758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4eb8fd0e5504d4cbeaca0d24d2dd7f4)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:24.155663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:24.202103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:24.321879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:24.458123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c96336eb4bfd461e82e7761077a5b910'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.459280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c96336eb4bfd461e82e7761077a5b910'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.486456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c96336eb4bfd461e82e7761077a5b910'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.499903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4eb8fd0e5504d4cbeaca0d24d2dd7f4'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.501404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4eb8fd0e5504d4cbeaca0d24d2dd7f4'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.511482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4eb8fd0e5504d4cbeaca0d24d2dd7f4'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.512480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4eb8fd0e5504d4cbeaca0d24d2dd7f4'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.514119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c96336eb4bfd461e82e7761077a5b910'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.514838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c96336eb4bfd461e82e7761077a5b910'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.528999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4b76da42-91c5-4edb-807d-e84da85e5662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4eb8fd0e5504d4cbeaca0d24d2dd7f4'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:24.615278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1019/2060] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:23 2022] DELETE /identity/v3/users/c96336eb4bfd461e82e7761077a5b910 => generated 0 bytes in 655 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:24.615278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:24.617416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:24.711764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e94b54-3908-4519-ae5c-9a01d1737d1b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:24.723628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1042/2061] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:24 2022] DELETE /identity/v3/users/f4eb8fd0e5504d4cbeaca0d24d2dd7f4 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:24.726465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.726862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.727295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.763912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:24.766168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:24.801363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:24.802021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:24.802301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:24.802576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] PATH_INFO: `/v3/users/5e9d4d3d07cd4ef08c791381d118f883` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:24.804128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9d4d3d07cd4ef08c791381d118f883)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:24.806397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:24.900424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1043/2062] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:24.917255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:24.919561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.016521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:25.040480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6555bd-4b81-43e1-b586-c5b2fab058b1 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.047093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.048181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.048910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.049313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.067365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.069024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.082645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1044/2063] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:25.084275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:25.085909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.101913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deac2ec9-25eb-4432-9693-8fb70e01492a service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.119954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.121506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.121984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.122404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] PATH_INFO: `/v3/projects/9cfcfe1ebb994061b6d203510fd7e65b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.123927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cfcfe1ebb994061b6d203510fd7e65b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.126313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.138864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9d4d3d07cd4ef08c791381d118f883'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.140340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9d4d3d07cd4ef08c791381d118f883'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.153059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9d4d3d07cd4ef08c791381d118f883'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.172454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9d4d3d07cd4ef08c791381d118f883'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.174107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3d96d123-4a96-4ac3-b1d2-ca1e9ff76734 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9d4d3d07cd4ef08c791381d118f883'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.307866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1020/2064] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:24 2022] DELETE /identity/v3/users/5e9d4d3d07cd4ef08c791381d118f883 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:25.313547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.314078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.314518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.339217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:25.341163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.344532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:25.381585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cfcfe1ebb994061b6d203510fd7e65b', 'request_id': 'req-e641a5f5-66ad-4f87-babc-f4b3b68febe7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.406420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cfcfe1ebb994061b6d203510fd7e65b', 'request_id': 'req-e641a5f5-66ad-4f87-babc-f4b3b68febe7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.415186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cfcfe1ebb994061b6d203510fd7e65b', 'request_id': 'req-e641a5f5-66ad-4f87-babc-f4b3b68febe7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.432921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1021/2065] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:25.450896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:25.454200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.520992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee7223-0be6-446b-8fa9-983fbb04f84c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.552094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] The token cache is being invalidate because project 9cfcfe1ebb994061b6d203510fd7e65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:25.552495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.553151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e641a5f5-66ad-4f87-babc-f4b3b68febe7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.561955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1045/2066] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:25 2022] DELETE /identity/v3/projects/9cfcfe1ebb994061b6d203510fd7e65b => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:25.565829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.566116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.566394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.594720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:25.597158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.598707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.599454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.599747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.600028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.602028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.691285 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1046/2067] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:25.696551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:25.697116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.697252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.697362 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8402463-235e-4532-a6fe-e5c1806be835 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.708552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1022/2068] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:25.710924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.711249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.712021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.731249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9174d7b8-2494-48fc-8f1b-24a95d9a599f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.738384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.738731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:25.739148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.739436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.739717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] PATH_INFO: `/v3/projects/26619c5e92de440eb5e812fcca3e37c2` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.741116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26619c5e92de440eb5e812fcca3e37c2)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.741269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:25.743876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.757699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1023/2069] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:25.763378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:25.775878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25b894-1c21-4597-8a1b-e40bba3ca5f5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.781729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.782437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.782729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.783000 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.789100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:25.789100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.789100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.799648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1024/2070] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:25.805146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:25.825003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9421481-6338-4b7e-8fc4-173d7cdadb5a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:25.832268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:25.832899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:25.833214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:25.833496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:25.834264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:25.836278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:25.928276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26619c5e92de440eb5e812fcca3e37c2', 'request_id': 'req-990c5f6d-1555-4bcc-9795-107ffac2c6a6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.937093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26619c5e92de440eb5e812fcca3e37c2', 'request_id': 'req-990c5f6d-1555-4bcc-9795-107ffac2c6a6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:25.943961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26619c5e92de440eb5e812fcca3e37c2', 'request_id': 'req-990c5f6d-1555-4bcc-9795-107ffac2c6a6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:26.037053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1025/2071] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:25 2022] POST /identity/v3/projects => generated 371 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:26.041404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.095289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:26.120309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4611784-b53c-4f2d-b8e5-9f9a5b50c4ef admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.126658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.127384 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.128092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.128524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.129194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.131238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.187197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] The token cache is being invalidate because project 26619c5e92de440eb5e812fcca3e37c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:26.187422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:26.188157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-990c5f6d-1555-4bcc-9795-107ffac2c6a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:26.189835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1047/2072] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:25 2022] DELETE /identity/v3/projects/26619c5e92de440eb5e812fcca3e37c2 => generated 0 bytes in 498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:26.195035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208b26-3766-4efa-87e0-96090772cdee None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.195350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208b26-3766-4efa-87e0-96090772cdee None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.195629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11208b26-3766-4efa-87e0-96090772cdee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.221106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-11208b26-3766-4efa-87e0-96090772cdee None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:26.222481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-11208b26-3766-4efa-87e0-96090772cdee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:26.232051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1048/2073] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:26.243672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.244170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.244708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.282957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:26.285033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:26.301692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1049/2074] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:26.312938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.319558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8480d-5752-47db-a349-6e28933d2858 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.326749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.327373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.327654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.327925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.328702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:26.328995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.330632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.345690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1050/2075] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:26.350295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.362895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead9779e-3643-4895-ae0d-bb0497bfd8f8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.368493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.369603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.370383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.372068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.372805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.374835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.437580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1026/2076] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:26 2022] POST /identity/v3/users => generated 286 bytes in 396 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:26.443184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.446192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:26.475112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ab1aa-5c9a-4c68-8406-f8983d4b319a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.484502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.485512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.485952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.486489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.487933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.490774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.520464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1027/2077] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:26.529747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.541657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1051/2078] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:26 2022] POST /identity/v3/projects => generated 373 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:26.543857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.551423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5c00c-9d66-4b67-9b1e-4854cedf8b52 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.564346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:26.653353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.653977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.654613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.654613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.655182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.656852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.657144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e1378b-5cab-4271-a746-769bb541b95d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.663377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.664008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.664330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.664615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.665258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.667248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.671501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1028/2079] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:26.675787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.703627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296ca30d-697e-44a4-b5d2-ef2733ac973a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.703627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.703627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.703627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.704125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] PATH_INFO: `/v3/projects/a9b5454a27cf4ddebff494a48d20586c/users/093622ce75c74f5388e03101ffe99998/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.712168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9b5454a27cf4ddebff494a48d20586c, user_id=093622ce75c74f5388e03101ffe99998, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.714964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.861915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1029/2080] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:26 2022] PUT /identity/v3/projects/a9b5454a27cf4ddebff494a48d20586c/users/093622ce75c74f5388e03101ffe99998/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:26.872434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.873123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1052/2081] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:26 2022] POST /identity/v3/users => generated 287 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:26.873620 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:26.873786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:26.888112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf16f6a2-297e-4385-a16c-76b7e65eb3f9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.954352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e059ba34-44e5-4309-91af-20665a8c7b3f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:26.960103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.960864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.961148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.961505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] PATH_INFO: `/v3/projects/4017be3f15894016a44bc4293f890268` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.962742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4017be3f15894016a44bc4293f890268)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.965575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:26.986972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:26.988027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:26.988580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:26.989084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:26.990294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:26.993539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac5fdde-2df4-4e44-8489-e3e3b925af35 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.013556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1053/2082] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:26 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:27.016080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.016854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.017557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.052170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] MFA Rules not processed for user `093622ce75c74f5388e03101ffe99998`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:27.058591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.139794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4017be3f15894016a44bc4293f890268', 'request_id': 'req-6110d8f5-786c-4d00-a005-cdef3e90b655'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.153329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1054/2083] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:27.154378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4017be3f15894016a44bc4293f890268', 'request_id': 'req-6110d8f5-786c-4d00-a005-cdef3e90b655'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.155209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.174020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4017be3f15894016a44bc4293f890268', 'request_id': 'req-6110d8f5-786c-4d00-a005-cdef3e90b655'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.174553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f127482-1b4b-4afb-afd7-165d7ac458b5 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.244312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.244980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.245267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.245543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.246280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.247940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.264761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1055/2084] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:27 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:27.264761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.267648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.270321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] The token cache is being invalidate because project 4017be3f15894016a44bc4293f890268 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:27.271381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.271381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.273151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1030/2085] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:26 2022] DELETE /identity/v3/projects/4017be3f15894016a44bc4293f890268 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:27.273938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.278430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.301116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6110d8f5-786c-4d00-a005-cdef3e90b655 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.311839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.312663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.313106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.313585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] PATH_INFO: `/v3/projects/1d8828b800304c70a009a8fe6902c6a9/users/292346f86de046b09f2572c879222b71/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.340969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d8828b800304c70a009a8fe6902c6a9, user_id=292346f86de046b09f2572c879222b71, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.346107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.385231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbb40c7-6973-4f99-aa47-432c07a4df87 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.391629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.392322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.392608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.392877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.394862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.466835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1031/2086] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:27 2022] PUT /identity/v3/projects/1d8828b800304c70a009a8fe6902c6a9/users/292346f86de046b09f2572c879222b71/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:27.467773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.473349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.475248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5245436-f252-4929-bd0f-6d03b5af9338 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.475536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.490658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1056/2087] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:27 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:27.492793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.493222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.496827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.546114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] MFA Rules not processed for user `292346f86de046b09f2572c879222b71`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:27.547579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.587495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:27.627122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea159d-df69-4cbe-accc-91854818539b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.647531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.648209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.648494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.648782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] PATH_INFO: `/v3/users/0a2ba434496e484c91fbec46cde4e7c1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.650340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a2ba434496e484c91fbec46cde4e7c1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.654230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.700886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1057/2088] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:27.733527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:27.768670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.771354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:27.877075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ebbca3-8064-4fbb-969e-e9dcefab001a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.881266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.881887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.882164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.882433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.892797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.895562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.907632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1058/2089] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:27 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:27.923179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a2ba434496e484c91fbec46cde4e7c1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.928231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a2ba434496e484c91fbec46cde4e7c1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.939773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:27.948380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a2ba434496e484c91fbec46cde4e7c1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.957611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f73bb9-7423-47ec-9d96-c4951399cec8 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:27.963994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:27.965464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:27.965857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:27.966303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] PATH_INFO: `/v3/users/292346f86de046b09f2572c879222b71` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:27.967881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] RBAC: Authorizing `identity:delete_user(user_id=292346f86de046b09f2572c879222b71)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:27.969918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a2ba434496e484c91fbec46cde4e7c1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:27.970552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:27.970882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6852317d-b781-45b4-a1d8-f9f1d69b58db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a2ba434496e484c91fbec46cde4e7c1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.090436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1032/2090] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:27 2022] DELETE /identity/v3/users/0a2ba434496e484c91fbec46cde4e7c1 => generated 0 bytes in 623 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:28.212407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292346f86de046b09f2572c879222b71'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.213355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292346f86de046b09f2572c879222b71'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.224260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292346f86de046b09f2572c879222b71'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.225191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292346f86de046b09f2572c879222b71'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.241939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292346f86de046b09f2572c879222b71'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:28.328344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1059/2091] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:27 2022] DELETE /identity/v3/users/292346f86de046b09f2572c879222b71 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:28.333982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:28.334401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:28.334965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:28.372134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:28.372134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:28.477239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1033/2092] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:28.490500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:28.492451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:28.592819 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92bb22-c5bb-4654-848d-927c1160fb27 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:28.597991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:28.598660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:28.598919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:28.599201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:28.610456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:28.613261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:28.627930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1060/2093] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:28.709712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:28.711536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:28.727779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea08cbc5-6a45-4fdd-9585-eae802207a20 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:28.736106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:28.741007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:28.741007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:28.741007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] PATH_INFO: `/v3/projects/beac19b524ae4d6a848caf5c96ca4709` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:28.741007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] RBAC: Authorizing `identity:delete_project(project_id=beac19b524ae4d6a848caf5c96ca4709)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:28.743537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:29.022870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beac19b524ae4d6a848caf5c96ca4709', 'request_id': 'req-f155bc9a-0d99-4203-ad5c-705335d1b864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.046677 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beac19b524ae4d6a848caf5c96ca4709', 'request_id': 'req-f155bc9a-0d99-4203-ad5c-705335d1b864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.056947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beac19b524ae4d6a848caf5c96ca4709', 'request_id': 'req-f155bc9a-0d99-4203-ad5c-705335d1b864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.236438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] The token cache is being invalidate because project beac19b524ae4d6a848caf5c96ca4709 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:29.237279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.238264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.240811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1034/2094] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:28 2022] DELETE /identity/v3/projects/beac19b524ae4d6a848caf5c96ca4709 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:29.246294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:29.257850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c9e26b-ceb3-4f32-a2d1-4245520b5f4e service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:29.330790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:29.331875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:29.332420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:29.332907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] PATH_INFO: `/v3/users/abb1c8ef24a74fb5b1f086d5306435c1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:29.335302 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=abb1c8ef24a74fb5b1f086d5306435c1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:29.340604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:29.367194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:29.385026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:29.386896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:29.407304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155bc9a-0d99-4203-ad5c-705335d1b864 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:29.414263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:29.420980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:29.420980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:29.420980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] PATH_INFO: `/v3/projects/1d8828b800304c70a009a8fe6902c6a9` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:29.422346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d8828b800304c70a009a8fe6902c6a9)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:29.424774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:29.624972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:29.642216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abb1c8ef24a74fb5b1f086d5306435c1'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.643102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abb1c8ef24a74fb5b1f086d5306435c1'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.653429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abb1c8ef24a74fb5b1f086d5306435c1'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.659129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d8828b800304c70a009a8fe6902c6a9', 'request_id': 'req-6d15785a-d1d8-44fb-991d-d752b83ea15a'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.659311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abb1c8ef24a74fb5b1f086d5306435c1'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.663882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d8828b800304c70a009a8fe6902c6a9', 'request_id': 'req-6d15785a-d1d8-44fb-991d-d752b83ea15a'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.671238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d8828b800304c70a009a8fe6902c6a9', 'request_id': 'req-6d15785a-d1d8-44fb-991d-d752b83ea15a'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.673507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d4516e81-817d-4749-814e-92284cdaae98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abb1c8ef24a74fb5b1f086d5306435c1'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.838834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] The token cache is being invalidate because project 1d8828b800304c70a009a8fe6902c6a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:29.840606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.842138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6d15785a-d1d8-44fb-991d-d752b83ea15a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:29.848279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1035/2095] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:29 2022] DELETE /identity/v3/projects/1d8828b800304c70a009a8fe6902c6a9 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:29.852033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:29.852571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:29.853079 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:29.895485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:29.897785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:29.970816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1061/2096] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:29 2022] DELETE /identity/v3/users/abb1c8ef24a74fb5b1f086d5306435c1 => generated 0 bytes in 726 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:30.016962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1036/2097] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:30.030842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.031382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.031953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.060518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:30.062823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:30.185660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1062/2098] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:30.191280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.208094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b755-e772-4aee-a182-00c6f01c15f5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.216634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.217390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.217690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.217951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.218646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:30.218983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.220761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.240202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1037/2099] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:30.244585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.259888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1b38e-8c07-4bbe-bcbd-529070b7ea3a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.266364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.266999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.267266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.267539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.268214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.270199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.421415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1063/2100] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:30 2022] POST /identity/v3/projects => generated 389 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:30.428811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.445281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40530f7d-823b-4f64-bf10-62da671a7667 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.545093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.545799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.546214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.546539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.547319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.549618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.656938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.664322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:30.675273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c25f410-66f7-4831-a821-c7012f2ef410 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.680957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.681624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.681927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.682202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] PATH_INFO: `/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.683834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6fdbf8d0cfa49b78c10f9c56958699d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.687608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.705002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1038/2101] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:30 2022] POST /identity/v3/users => generated 295 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:30.706455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.726229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edc0712-d3d1-4d4b-a3c3-3d69179b2618 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.738790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.739829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.740120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.740423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.741275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.743087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.760067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1039/2102] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:30.765260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.776686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabe3e04-5304-49b6-9c63-d171b11f9e95 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.782531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.783285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.783657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.784022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.784891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.786605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.798530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1040/2103] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:30.805214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:30.818273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300eeccf-8638-499d-9740-8c30c6275414 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:30.826046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:30.826046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.826046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.826046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] PATH_INFO: `/v3/projects/86f4ebeaa21949a1b2736cba21a26d1c/users/491fb54f385e4f79bbaf6f9895d69997/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.845305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] RBAC: Authorizing `identity:create_grant(project_id=86f4ebeaa21949a1b2736cba21a26d1c, user_id=491fb54f385e4f79bbaf6f9895d69997, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:30.848170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaece23-054d-4d42-ba46-633506d396df admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:30.891184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:30.965128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1041/2104] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:30 2022] PUT /identity/v3/projects/86f4ebeaa21949a1b2736cba21a26d1c/users/491fb54f385e4f79bbaf6f9895d69997/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:30.972215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:30.972782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:30.973277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:30.987995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6fdbf8d0cfa49b78c10f9c56958699d', 'request_id': 'req-8cd15082-fea0-4599-a3e2-1c48fb962f4e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:31.007724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6fdbf8d0cfa49b78c10f9c56958699d', 'request_id': 'req-8cd15082-fea0-4599-a3e2-1c48fb962f4e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:31.019676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6fdbf8d0cfa49b78c10f9c56958699d', 'request_id': 'req-8cd15082-fea0-4599-a3e2-1c48fb962f4e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:31.053403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] MFA Rules not processed for user `491fb54f385e4f79bbaf6f9895d69997`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:31.055746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.120727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:31.153932 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1042/2105] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:30 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:31.171437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.173813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.218837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] The token cache is being invalidate because project a6fdbf8d0cfa49b78c10f9c56958699d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:31.218837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:31.218837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-8cd15082-fea0-4599-a3e2-1c48fb962f4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:31.218837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1064/2106] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:30 2022] DELETE /identity/v3/projects/a6fdbf8d0cfa49b78c10f9c56958699d => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:31.220994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.221724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.222046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.251205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:31.253332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.254920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ef1cdb-8bee-48b8-8b79-a74cd558ee75 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.262200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.263651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.264900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.264900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.268408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.289380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.290921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e998136c-6c64-437a-8e61-2785af72b208 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.303014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1043/2107] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:31 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:31.391441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1065/2108] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:31.401517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.401962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.402931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.437111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:31.438334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.447445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1044/2109] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:31.453212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.483565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed257459-921b-465c-a029-32ba0ed35d47 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.497868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.505467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.534429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1066/2110] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:03:31.538762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.548754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d47d3cb-a3e8-4b57-8798-b325151f2d49 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.554461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.555077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.555358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.555627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.556328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.559129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.662915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.662915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:31.699672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db0712-1b25-4058-9be0-de9a05eea052 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.701125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1045/2111] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:31 2022] POST /identity/v3/projects => generated 397 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:31.705625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:31.791454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:31.810602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.811408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.811787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.812205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] PATH_INFO: `/v3/users/093622ce75c74f5388e03101ffe99998` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.813864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] RBAC: Authorizing `identity:delete_user(user_id=093622ce75c74f5388e03101ffe99998)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.818922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:31.828771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cd526-2141-4b1b-97c6-a1a36a5143a7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:31.834017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:31.834644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:31.834929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:31.835202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:31.835843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:31.837918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.017803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1046/2112] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:31 2022] POST /identity/v3/users => generated 299 bytes in 316 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:32.022175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:32.032627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0315b7a7-bf4b-4785-a664-1ec6c25f3603 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:32.037415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:32.038081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.038372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.038648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.041307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:32.041307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.052457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1047/2113] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:32.056423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:32.067517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e0f135-62db-4a3d-92ae-8817d3bff499 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:32.072511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:32.073160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.073446 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.073725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.074406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:32.076008 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.091267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1048/2114] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:32 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:32.099121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:32.122013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f6d9e-4027-4fb5-8f65-f0e6748b3251 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:32.127654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:32.128754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.129224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.129689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] PATH_INFO: `/v3/projects/0b443d2af7944dd18b6325287ec99c46/users/b95f3c3893e24a9f9fd7e909eb651f46/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.157417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b443d2af7944dd18b6325287ec99c46, user_id=b95f3c3893e24a9f9fd7e909eb651f46, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:32.161252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8742a5b6-d949-42d7-a111-c30b19a3a2dd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.166552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093622ce75c74f5388e03101ffe99998'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:32.167431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093622ce75c74f5388e03101ffe99998'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:32.186370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093622ce75c74f5388e03101ffe99998'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:32.187484 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093622ce75c74f5388e03101ffe99998'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:32.208459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d593328e-98e0-41c3-aed9-2428fe28054a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093622ce75c74f5388e03101ffe99998'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:32.275308 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1049/2115] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:32 2022] PUT /identity/v3/projects/0b443d2af7944dd18b6325287ec99c46/users/b95f3c3893e24a9f9fd7e909eb651f46/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:32.290170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37513-f196-4316-b287-d3128797e179 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.290547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37513-f196-4316-b287-d3128797e179 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.290864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d37513-f196-4316-b287-d3128797e179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.327631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a5d37513-f196-4316-b287-d3128797e179 None None] MFA Rules not processed for user `b95f3c3893e24a9f9fd7e909eb651f46`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:32.328657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1067/2116] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:31 2022] DELETE /identity/v3/users/093622ce75c74f5388e03101ffe99998 => generated 0 bytes in 670 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:32.333640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a5d37513-f196-4316-b287-d3128797e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:32.335218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.335505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.335880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.358831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:32.362681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:32.366586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:32.448211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1050/2117] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:32.466842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d37513-f196-4316-b287-d3128797e179 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:32.469912 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a5d37513-f196-4316-b287-d3128797e179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:32.490576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1068/2118] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:32.509850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:32.513290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:32.604220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d37513-f196-4316-b287-d3128797e179 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:32.611771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:32.612819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.612984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.613356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.624647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:32.626208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.636449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1051/2119] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:32.660265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24dc32d-e320-443d-9e7a-811b0d708970 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:32.666872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:32.668109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:32.675251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:32.675251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:32.687263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:32.690361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:32.708272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1069/2120] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:33.142527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:33.307492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:33.309839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:33.317415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:33.332698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9440d95-e484-4a1a-af7e-1d7c3f77768e service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:33.337954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:33.338627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:33.338871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:33.339146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] PATH_INFO: `/v3/users/fce05334b66946f49fefc4e8b36f2c9c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:33.340377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fce05334b66946f49fefc4e8b36f2c9c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:33.342295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:33.399154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:33.401348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:33.432284 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d301dc6a-31f3-49b1-bb31-2a2088cc65df service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:33.450697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:33.451424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:33.451817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:33.452326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] PATH_INFO: `/v3/projects/a9b5454a27cf4ddebff494a48d20586c` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:33.455704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9b5454a27cf4ddebff494a48d20586c)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:33.468439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:33.644130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fce05334b66946f49fefc4e8b36f2c9c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.646648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fce05334b66946f49fefc4e8b36f2c9c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.686700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9b5454a27cf4ddebff494a48d20586c', 'request_id': 'req-c337171f-6c40-466d-a842-6fac721c9f8e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.693095 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fce05334b66946f49fefc4e8b36f2c9c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.699077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9b5454a27cf4ddebff494a48d20586c', 'request_id': 'req-c337171f-6c40-466d-a842-6fac721c9f8e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.706704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9b5454a27cf4ddebff494a48d20586c', 'request_id': 'req-c337171f-6c40-466d-a842-6fac721c9f8e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.746473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fce05334b66946f49fefc4e8b36f2c9c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.747683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:33.749459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3caa169e-bf6e-4715-a21a-c87ba20199a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fce05334b66946f49fefc4e8b36f2c9c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.875224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:33.884065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] The token cache is being invalidate because project a9b5454a27cf4ddebff494a48d20586c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:33.884458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.885188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c337171f-6c40-466d-a842-6fac721c9f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:33.886831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1070/2121] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:33 2022] DELETE /identity/v3/projects/a9b5454a27cf4ddebff494a48d20586c => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:33.891941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ebe6-e7be-4ee8-b052-272c6482a1d8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:33.892341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ebe6-e7be-4ee8-b052-272c6482a1d8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:33.892623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd84ebe6-e7be-4ee8-b052-272c6482a1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:33.921859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-cd84ebe6-e7be-4ee8-b052-272c6482a1d8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:33.923118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-cd84ebe6-e7be-4ee8-b052-272c6482a1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.010099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1071/2122] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:34.021436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.022402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.022402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.042389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1052/2123] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:33 2022] DELETE /identity/v3/users/fce05334b66946f49fefc4e8b36f2c9c => generated 0 bytes in 735 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:34.048579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.048929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.049486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.051027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:34.052767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.076875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:34.078273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.174771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1072/2124] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:34.176513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.196385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a5c2b-a260-4439-a86c-13e86d2bc9a0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.196688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1053/2125] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:34.206411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.206648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.207754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.208461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:34.209141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.209631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.211702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:34.212269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.216538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.244690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1073/2126] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:34.250266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.271088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9edb66-d2c2-4c48-970e-25a63034b66e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.277022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.278294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.278679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.279089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.280240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.282311 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.357862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b67ba4-7435-4893-9b74-773bb8ded283 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.369943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.369943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.369943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.369943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.376617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.378374 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.391634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1054/2127] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:34.453685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1074/2128] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:34 2022] POST /identity/v3/projects => generated 405 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:34.454719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.472823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affca9fb-b15d-48ac-b4eb-38dae7a7d960 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.620030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.620696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.621516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.621801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.622459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.624589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.725309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1055/2129] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:34 2022] POST /identity/v3/users => generated 303 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:34.730523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.744530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a586233-7a0c-40b7-93b2-ffda6b000555 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.751525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.752198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.753553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.753553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.754152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.755805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.769581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1075/2130] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:34.774259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.789063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83de5122-47bf-4bd9-a863-36fe145f920d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.795209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.796094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.796470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.797011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.797574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.799576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:34.816917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1056/2131] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:34 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:34.825057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:34.841312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007b7a79-92e4-4b53-b176-9274b1a90700 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:34.847866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:34.848801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:34.850380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:34.850380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] PATH_INFO: `/v3/projects/48231b60b20f4766a1713f1535dcbf91/users/9a678635e10344ac85d0d6c3d15cfbae/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:34.875522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] RBAC: Authorizing `identity:create_grant(project_id=48231b60b20f4766a1713f1535dcbf91, user_id=9a678635e10344ac85d0d6c3d15cfbae, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:34.878752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54486c3-bd01-4805-84b6-cdf111cf3acf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:35.004208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:35.006393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:35.024201 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca51912d-a469-4ec4-b18c-74f382539925 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:35.039817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1076/2132] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:34 2022] PUT /identity/v3/projects/48231b60b20f4766a1713f1535dcbf91/users/9a678635e10344ac85d0d6c3d15cfbae/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:35.048391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.048913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.049440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.077930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:35.079200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:35.110073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:35.110715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.111001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.111281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] PATH_INFO: `/v3/projects/550a0d254a4f4d6c9d5645bf2a9fb0fa` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.112626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=550a0d254a4f4d6c9d5645bf2a9fb0fa)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:35.114892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:35.118527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:35.171398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1077/2133] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:35.182616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:35.185037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:35.238497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '550a0d254a4f4d6c9d5645bf2a9fb0fa', 'request_id': 'req-3611223b-a5d8-411a-899c-70ef3a64a4b5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:35.246960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '550a0d254a4f4d6c9d5645bf2a9fb0fa', 'request_id': 'req-3611223b-a5d8-411a-899c-70ef3a64a4b5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:35.259195 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '550a0d254a4f4d6c9d5645bf2a9fb0fa', 'request_id': 'req-3611223b-a5d8-411a-899c-70ef3a64a4b5'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:35.278964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21dffe1-17b4-4f1f-91d4-79c893cff634 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:35.359110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:35.359722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.360007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.360499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.389893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] The token cache is being invalidate because project 550a0d254a4f4d6c9d5645bf2a9fb0fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:35.389893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:35.389893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3611223b-a5d8-411a-899c-70ef3a64a4b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:35.393109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1057/2134] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:35 2022] DELETE /identity/v3/projects/550a0d254a4f4d6c9d5645bf2a9fb0fa => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:35.399143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.400037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.400814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.456925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:35.459007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:35.486762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:35.488558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bf239-83c0-4023-b350-f10399de3460 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:35.507461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1078/2135] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:35.550305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1058/2136] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:35.563278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.563600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.563835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.601643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:35.603606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:35.616203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1079/2137] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:35.618439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:35.636436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f87cf2-686e-4802-813f-67eac6ed7931 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:35.643883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:35.644624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.644976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.645332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.646085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:35.646458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:35.648212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:35.663435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1059/2138] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:35.669660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:35.679466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e972bfc-6c75-4ed5-b1c3-57766696ca2f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:35.687082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:35.689322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.689322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.689322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.690440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:35.694298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:35.892872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1080/2139] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:35 2022] POST /identity/v3/projects => generated 361 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:35.897221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:35.911523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd64e5b-5777-4860-8160-d91e1b6894c4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:35.974140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:35.974755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:35.975032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:35.975307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:35.975951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:35.977894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.140836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1060/2140] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:35 2022] POST /identity/v3/users => generated 281 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:36.145958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:36.161532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727a155-e044-412e-bbbd-02550afde3df admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:36.167947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:36.168741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.169024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.169295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.169988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:36.171787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.183167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1081/2141] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:36.188066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:36.199193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e62dfc4-2881-400b-b7ef-99e9c02108ca admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:36.204218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:36.204882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.205164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.205418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.206172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:36.207940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e36f784-4d6a-4cc1-8b44-5c680868c03d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.221430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1061/2142] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:36.225939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:36.236562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:36.237048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9043c7b3-d06f-40d6-b761-9fa60e5c36e3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:36.241959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:36.242670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.246538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.246538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] PATH_INFO: `/v3/projects/264f5116fdc14dfea552965bf6ef8d13/users/abab48397d2f4f15bbe03740feafb5a2/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.265980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] RBAC: Authorizing `identity:create_grant(project_id=264f5116fdc14dfea552965bf6ef8d13, user_id=abab48397d2f4f15bbe03740feafb5a2, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:36.268769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.401788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1082/2143] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:36 2022] PUT /identity/v3/projects/264f5116fdc14dfea552965bf6ef8d13/users/abab48397d2f4f15bbe03740feafb5a2/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:36.415935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.416671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.416671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.453880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] MFA Rules not processed for user `abab48397d2f4f15bbe03740feafb5a2`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:36.455438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:36.550742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1062/2144] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:36.618509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:36.620632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:36.738234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5951d9a-352d-4030-b14c-54ddfb18a006 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:36.743789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:36.744649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.744993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.745334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.757131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:36.759123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30639ec5-a612-423e-9cf5-4671a25e88a1 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.771784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1083/2145] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:36 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:36.796200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:36.898930 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f8d32a-fa04-49a9-acfc-4671ce0a181b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:36.904595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:36.905356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:36.905641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:36.905918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] PATH_INFO: `/v3/users/abab48397d2f4f15bbe03740feafb5a2` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:36.907226 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] RBAC: Authorizing `identity:delete_user(user_id=abab48397d2f4f15bbe03740feafb5a2)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:36.909236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:36.954188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:37.204615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abab48397d2f4f15bbe03740feafb5a2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:37.205791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abab48397d2f4f15bbe03740feafb5a2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:37.220340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abab48397d2f4f15bbe03740feafb5a2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:37.244830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abab48397d2f4f15bbe03740feafb5a2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:37.246633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abab48397d2f4f15bbe03740feafb5a2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:37.344541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1063/2146] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:36 2022] DELETE /identity/v3/users/abab48397d2f4f15bbe03740feafb5a2 => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:37.349663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b93e-5c2a-4f1b-9015-f68cb4ba9fb4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:37.349995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b93e-5c2a-4f1b-9015-f68cb4ba9fb4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:37.350365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b4b93e-5c2a-4f1b-9015-f68cb4ba9fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:37.376377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-10b4b93e-5c2a-4f1b-9015-f68cb4ba9fb4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:37.376377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-10b4b93e-5c2a-4f1b-9015-f68cb4ba9fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:37.447430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1084/2147] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:37.458638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:37.461034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:37.533080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:37.561013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08436-77bf-4f5d-b6a2-5ba62377024b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:37.572459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:37.572747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:37.573024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:37.577673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:37.578547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:37.578964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:37.579323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:37.594926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:37.599515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:37.612857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1064/2148] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:37.618520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] MFA Rules not processed for user `9a678635e10344ac85d0d6c3d15cfbae`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:37.619875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:37.708756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1085/2149] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:37.712218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:37.713810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:37.729294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53256f58-e2a8-44fc-a3d5-05f45f156aad service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:37.743242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:37.745046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:37.748792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:37.748792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:37.748792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:37.748792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:37.749634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:37.896971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1065/2150] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:37 2022] POST /identity/v3/projects => generated 403 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:37.898573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:37.923026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322647fb-d5fe-4852-a925-94a9065ef674 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.006193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.007213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.007213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.007513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.007979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.010055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.146314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1086/2151] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:37 2022] POST /identity/v3/users => generated 308 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:38.150417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.162050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba9a1f-5c7b-4bbb-81fd-260ab3f16275 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.167318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.167990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.168307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.168585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.169244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.171181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.182967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1066/2152] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:38.194931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.198142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8a4351-df0a-49c5-845b-7e8966d41769 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.203605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.204634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.204803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.205105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] PATH_INFO: `/v3/projects/63713ff07af742e79def84f6e760f8e5/users/bb76e2bed6094e9a8098506d861efb64/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.228569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63713ff07af742e79def84f6e760f8e5, user_id=bb76e2bed6094e9a8098506d861efb64, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.231456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.273289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.275832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.292392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48fa3734-c464-4f76-a98d-8a443a46f11c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.299224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.299965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.300174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.300478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] PATH_INFO: `/v3/projects/264f5116fdc14dfea552965bf6ef8d13` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.301640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264f5116fdc14dfea552965bf6ef8d13)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.303563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.397843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1087/2153] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:38 2022] PUT /identity/v3/projects/63713ff07af742e79def84f6e760f8e5/users/bb76e2bed6094e9a8098506d861efb64/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:38.402336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.455464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264f5116fdc14dfea552965bf6ef8d13', 'request_id': 'req-ffe04ba8-58c2-409a-8e46-4c2605d18542'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:38.465649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264f5116fdc14dfea552965bf6ef8d13', 'request_id': 'req-ffe04ba8-58c2-409a-8e46-4c2605d18542'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:38.473794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264f5116fdc14dfea552965bf6ef8d13', 'request_id': 'req-ffe04ba8-58c2-409a-8e46-4c2605d18542'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:38.487835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa04c2b-df8b-4745-a29e-6b5e96854c07 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.557725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.558737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.559240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.559982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.561222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.564349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.586785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] The token cache is being invalidate because project 264f5116fdc14dfea552965bf6ef8d13 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:38.588707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:38.588707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ffe04ba8-58c2-409a-8e46-4c2605d18542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:38.590439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1088/2154] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:38 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:38.590749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1067/2155] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:38 2022] DELETE /identity/v3/projects/264f5116fdc14dfea552965bf6ef8d13 => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:38.591436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.593316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.599218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.599521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.599811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.609219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12926445-9b74-49ab-85f4-ca89f0713609 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.614398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.615442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.615442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.615658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] PATH_INFO: `/v3/projects/63713ff07af742e79def84f6e760f8e5/users/bb76e2bed6094e9a8098506d861efb64/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.617855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] RBAC: Authorizing `identity:create_grant(project_id=63713ff07af742e79def84f6e760f8e5, user_id=bb76e2bed6094e9a8098506d861efb64, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.620642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcba0f8-6b18-42fa-978b-29de97c3478b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.623242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:38.625214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.632978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1089/2156] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:38 2022] PUT /identity/v3/projects/63713ff07af742e79def84f6e760f8e5/users/bb76e2bed6094e9a8098506d861efb64/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:38.694623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:38.715343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1068/2157] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:38.725693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.725693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.725944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.776517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:38.778644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:38.789864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1090/2158] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:38.800383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.811617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85916dc5-3751-426b-ad8c-62adcdd8150d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.816849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.817474 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.817758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.818036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.818731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:38.819039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.820721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.837919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1069/2159] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:38.842938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:38.852382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e0edf8-bac5-4394-abf5-013f63c9c4c2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:38.858041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:38.858740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:38.859025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:38.859310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:38.859938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:38.862026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:38.990011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.029431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1091/2160] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:38 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:39.034505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:39.046549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40eb893d-a2b3-40c1-8e75-debb4f9c4f47 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:39.129458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:39.130844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.131210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.131449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.132100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:39.134831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:39.183012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.270261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1070/2161] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:39 2022] POST /identity/v3/users => generated 292 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:39.273108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:39.289592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c347a6ea-0684-405a-bdde-b809fddbdf9d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:39.295290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:39.295933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.296249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.296658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.297333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:39.299238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:39.311984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1092/2162] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:39.316646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:39.337281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00a7226-e276-42b0-bf5d-230dbc7ea5d6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:39.337281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:39.337281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.337281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.338105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.338105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:39.338105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708d8090-192b-4fc3-b950-b93a1459bdb1 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:39.350822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1071/2163] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:39.354393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:39.367680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fcb2e5-42a5-4c8b-ad30-8271091c26a6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:39.373591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:39.374439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.375072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.375751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] PATH_INFO: `/v3/projects/9aa00aef04784c1ea61221c69e872045/users/775ee674f5084d01b48d7b3ef20c7864/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.409606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9aa00aef04784c1ea61221c69e872045, user_id=775ee674f5084d01b48d7b3ef20c7864, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:39.415588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:39.566610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1093/2164] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:39 2022] PUT /identity/v3/projects/9aa00aef04784c1ea61221c69e872045/users/775ee674f5084d01b48d7b3ef20c7864/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:39.573223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5a7e-7314-4d4b-ac7e-2b523a202906 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.573767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5a7e-7314-4d4b-ac7e-2b523a202906 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.574635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a5a7e-7314-4d4b-ac7e-2b523a202906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.588269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.766238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.768310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-992a5a7e-7314-4d4b-ac7e-2b523a202906 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:39.769890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-992a5a7e-7314-4d4b-ac7e-2b523a202906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:39.861916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1072/2165] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:39.887656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:39.889729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:39.937083 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.961404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:39.979699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bd82e2-89c8-4bad-820c-7436dc0c26cc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:39.983989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:39.984711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:39.984994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:39.985270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:39.996985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:40.001192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:40.002658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:40.014128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1094/2166] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:03:40.400338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:40.400999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:40.403009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:40.474920 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] MFA Rules not processed for user `bb76e2bed6094e9a8098506d861efb64`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:40.476892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:40.598101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1073/2167] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:40 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:40.741467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:41.195978 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.516962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:42.528677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b52461-9ae5-47f3-b42a-1f497205490e service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:42.534248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:42.539953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:42.539953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:42.539953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] PATH_INFO: `/v3/users/9a678635e10344ac85d0d6c3d15cfbae` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:42.539953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a678635e10344ac85d0d6c3d15cfbae)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:42.539953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:42.624375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:42.850706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a678635e10344ac85d0d6c3d15cfbae'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.853077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a678635e10344ac85d0d6c3d15cfbae'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.870347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a678635e10344ac85d0d6c3d15cfbae'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.871977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a678635e10344ac85d0d6c3d15cfbae'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:42.899929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a678635e10344ac85d0d6c3d15cfbae'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.032181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:43.052299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1095/2168] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:42 2022] DELETE /identity/v3/users/9a678635e10344ac85d0d6c3d15cfbae => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:43.646524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:43.670279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284e99c0-f4f6-4463-8058-eed225631bed None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:43.743247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:43.743975 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:43.744356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:43.744985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] PATH_INFO: `/v3/projects/48231b60b20f4766a1713f1535dcbf91` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:43.746273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48231b60b20f4766a1713f1535dcbf91)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:43.748346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:43.936857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48231b60b20f4766a1713f1535dcbf91', 'request_id': 'req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.952961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48231b60b20f4766a1713f1535dcbf91', 'request_id': 'req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:43.965488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48231b60b20f4766a1713f1535dcbf91', 'request_id': 'req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.069737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] The token cache is being invalidate because project 48231b60b20f4766a1713f1535dcbf91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:44.070050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.071133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.075276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1074/2169] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:43 2022] DELETE /identity/v3/projects/48231b60b20f4766a1713f1535dcbf91 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:44.081110 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:44.082760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:44.100885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec786f-3204-4042-a089-d2a0e19ec88c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:44.191380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:44.193713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:44.194170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:44.195016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] PATH_INFO: `/v3/users/bb76e2bed6094e9a8098506d861efb64` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:44.197000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb76e2bed6094e9a8098506d861efb64)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:44.201305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:44.490689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb76e2bed6094e9a8098506d861efb64'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.492221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb76e2bed6094e9a8098506d861efb64'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.508571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb76e2bed6094e9a8098506d861efb64'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.510137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb76e2bed6094e9a8098506d861efb64'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.533244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:44.544338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-50e36516-3043-4214-9732-0480d5fb58be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb76e2bed6094e9a8098506d861efb64'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:44.648844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:44.716111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1096/2170] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:44 2022] DELETE /identity/v3/users/bb76e2bed6094e9a8098506d861efb64 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:45.255754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:45.258568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:45.307616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1f5dfc-0f94-4a1b-bf42-308e954f1ae4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:45.398861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:45.476185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:45.477340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:45.477817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:45.490042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] PATH_INFO: `/v3/projects/63713ff07af742e79def84f6e760f8e5` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:45.490042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] RBAC: Authorizing `identity:delete_project(project_id=63713ff07af742e79def84f6e760f8e5)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:45.490042 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:45.664136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63713ff07af742e79def84f6e760f8e5', 'request_id': 'req-57c148c2-78db-486d-bd89-83f8db21476c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:45.684673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63713ff07af742e79def84f6e760f8e5', 'request_id': 'req-57c148c2-78db-486d-bd89-83f8db21476c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:45.695634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63713ff07af742e79def84f6e760f8e5', 'request_id': 'req-57c148c2-78db-486d-bd89-83f8db21476c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:45.832361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] The token cache is being invalidate because project 63713ff07af742e79def84f6e760f8e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:45.833781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:45.835132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-57c148c2-78db-486d-bd89-83f8db21476c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:45.843501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1075/2171] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:45 2022] DELETE /identity/v3/projects/63713ff07af742e79def84f6e760f8e5 => generated 0 bytes in 583 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:45.844706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:45.845399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:45.845399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:45.886968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:45.886968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:46.032694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1097/2172] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:46.045356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.046184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.046879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.087571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:46.107082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:46.108517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:46.121942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1076/2173] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:46.130829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.138326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7492d4a6-8f30-4fe5-b43a-387a8228f8f9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.144492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.145396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.145833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.146257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.147103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:46.147906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.149900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.165572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1098/2174] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:46.171512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.190376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623871c9-3630-46fa-9a2f-0ba9dfa73d43 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.201863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.203069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.203643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.204473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.206143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.212131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.343012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1077/2175] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:46 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:46.347477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.368246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a748b-eeb2-4e6b-8537-29500c97cb39 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.488996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.489689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.489959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.490420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.491154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.493287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.625839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1099/2176] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:46 2022] POST /identity/v3/users => generated 285 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:46.627884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.642438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45a9cb7-5428-4e76-bb30-698497a2dd38 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.652834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.654203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.654643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.655053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.655862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.657924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.670484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1078/2177] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:46.677055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.703731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c01364-c202-4550-a0d8-a4eed3e6cd44 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.711863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.713096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.713647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.714146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.716165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.719389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb18028-f75c-4d3a-b969-347baca54901 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.736647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1100/2178] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:46.741699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:46.755988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41760dfc-d196-4be1-81a1-07253f0eee4a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:46.764733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:46.765938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.766448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.766940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] PATH_INFO: `/v3/projects/71fc374a856a4043ba26051cba4ff20c/users/a61748468b1245fc83f54b74f06e744e/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.799516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=71fc374a856a4043ba26051cba4ff20c, user_id=a61748468b1245fc83f54b74f06e744e, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:46.806147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:46.920165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1079/2179] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:46 2022] PUT /identity/v3/projects/71fc374a856a4043ba26051cba4ff20c/users/a61748468b1245fc83f54b74f06e744e/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:46.935319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:46.935319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:46.935319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:46.979944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] MFA Rules not processed for user `a61748468b1245fc83f54b74f06e744e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:46.981812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:47.089133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1101/2180] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:47.094755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:47.167772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f0d8a2-b57e-4c07-9e0a-01a50cd60cdc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:47.170888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:47.175412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:47.176130 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:47.176365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:47.176630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:47.177277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:47.179345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:47.294564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1080/2181] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:47 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:47.300426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:47.326802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb85e32-349b-4bc8-b897-7f3a58c6117b None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:47.428077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:47.428945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:47.429236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:47.429510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:47.430364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:47.432734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:47.536234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1102/2182] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:47 2022] POST /identity/v3/users => generated 290 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:47.537846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:47.552628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4d2868-c901-4a37-9afb-8e365f03516e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:47.560995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:47.562046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:47.562398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:47.563952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:47.564910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:47.566815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:47.581338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1081/2183] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:47.585805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:47.604272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e17aad3-ba19-4532-b064-4e5f1f32e8ee admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:47.611690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:47.612556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:47.613036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:47.613755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] PATH_INFO: `/v3/projects/1a4eb29cbc0d45038b1686022ba628a7/users/a6f59ef659f3482f85b7c30d3fcc73c3/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:47.635242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4eb29cbc0d45038b1686022ba628a7, user_id=a6f59ef659f3482f85b7c30d3fcc73c3, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:47.636854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9982a5a0-34f9-444a-92dc-1e38f4da6684 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.048809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1103/2184] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:47 2022] PUT /identity/v3/projects/1a4eb29cbc0d45038b1686022ba628a7/users/a6f59ef659f3482f85b7c30d3fcc73c3/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:48.052871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.066818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63288612-6553-47e5-8a62-25ec1775d544 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.155897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.156233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.156575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.170576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.171779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.172309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.172792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.173741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.176960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.196327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1082/2185] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:48.198583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.199461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] MFA Rules not processed for user `775ee674f5084d01b48d7b3ef20c7864`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:48.201069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.211064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf414b-17a8-435a-bb03-5632d47e3366 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.216649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.217413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.217612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.218115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] PATH_INFO: `/v3/projects/1a4eb29cbc0d45038b1686022ba628a7/users/a6f59ef659f3482f85b7c30d3fcc73c3/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.218291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:48.220400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a4eb29cbc0d45038b1686022ba628a7, user_id=a6f59ef659f3482f85b7c30d3fcc73c3, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.223078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991579ed-6891-45a3-9683-8dc6c1de9df9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.236428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1083/2186] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:48 2022] PUT /identity/v3/projects/1a4eb29cbc0d45038b1686022ba628a7/users/a6f59ef659f3482f85b7c30d3fcc73c3/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:48.255832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.256120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.261172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.306849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] MFA Rules not processed for user `a6f59ef659f3482f85b7c30d3fcc73c3`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:48.308848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.311820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1104/2187] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:48.322131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.331549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.347798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd63025-3c30-4379-a912-ce80c91449a3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.414537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1084/2188] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:48 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:48.434427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.436636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:48.439201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.440569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.441410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.441784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.443020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.446821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.536878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96271ff0-3e0e-487a-957b-e0ac6d35e93e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.541851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.542824 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.543299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.543922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.554965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.556523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.568846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1085/2189] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:48 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:48.599275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1105/2190] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:48 2022] POST /identity/v3/projects => generated 383 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:48.603410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.623806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a631c-cb23-489a-b10d-3c9edc87657b service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.713973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.715079 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.715588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.716077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.717483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.721314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.746612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:48.848569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1086/2191] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:48 2022] POST /identity/v3/users => generated 298 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:48.852851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.877667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7d38d4-fcb9-4dcb-b44d-f2d4a09ec311 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.887877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.900788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.901197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.901577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:48.902759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:48.904633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:48.927732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1106/2192] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:48 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:48.932698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:48.938914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:48.956045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c615f6c-84f1-4241-9041-824942059a27 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:48.966802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:48.968288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:48.969036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:48.970029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] PATH_INFO: `/v3/projects/df63dfc21ddb4ed596a16254145a59e4/users/5d8edf055ab9430ca2af83e70ad27f66/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:49.000043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df63dfc21ddb4ed596a16254145a59e4, user_id=5d8edf055ab9430ca2af83e70ad27f66, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:49.002788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:49.077128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1087/2193] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:48 2022] PUT /identity/v3/projects/df63dfc21ddb4ed596a16254145a59e4/users/5d8edf055ab9430ca2af83e70ad27f66/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:49.081683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:49.094372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8744f23c-0ff7-457c-8e96-e5378cc3047e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:49.183857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:49.184519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:49.184798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:49.185068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:49.185725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:49.187354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:49.205434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1107/2194] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:49 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:03:49.206744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:49.218169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bd57c9-8e34-4196-9072-fac71f0f3b11 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:49.224287 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:49.224967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:49.225250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:49.225522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] PATH_INFO: `/v3/projects/df63dfc21ddb4ed596a16254145a59e4/users/5d8edf055ab9430ca2af83e70ad27f66/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:49.228952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df63dfc21ddb4ed596a16254145a59e4, user_id=5d8edf055ab9430ca2af83e70ad27f66, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:49.234646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e742314-7307-4e5a-8f48-b134bb69bb40 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:49.253840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1088/2195] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:49 2022] PUT /identity/v3/projects/df63dfc21ddb4ed596a16254145a59e4/users/5d8edf055ab9430ca2af83e70ad27f66/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:49.606142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:49.727064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:50.095337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:50.326901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:50.881106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:51.395088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:51.804660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:51.809929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:51.838036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ea6b0-ee57-48f4-8a6e-035059687ebf admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:51.984432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:51.987680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:51.987680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:51.987680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] PATH_INFO: `/v3/users/b95f3c3893e24a9f9fd7e909eb651f46` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:51.991619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b95f3c3893e24a9f9fd7e909eb651f46)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:51.996030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:52.323815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b95f3c3893e24a9f9fd7e909eb651f46'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:52.326901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b95f3c3893e24a9f9fd7e909eb651f46'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:52.348372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b95f3c3893e24a9f9fd7e909eb651f46'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:52.351078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b95f3c3893e24a9f9fd7e909eb651f46'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:52.381904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b95f3c3893e24a9f9fd7e909eb651f46'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:52.571699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1108/2196] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:51 2022] DELETE /identity/v3/users/b95f3c3893e24a9f9fd7e909eb651f46 => generated 0 bytes in 769 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:52.578174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.580015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.580956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.633846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:52.633846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.779668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1089/2197] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:03:52.788423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:52.798069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:52.962428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbc34b4-0225-454c-8140-bb43de21b49c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:52.969136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:52.974634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:52.974634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:52.974634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:52.982507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:52.984427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a27b52a-3dfb-44a9-aa9c-1e140f3f3036 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:52.997223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1109/2198] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:53.122445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:53.865025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:53.903582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:53.905962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:53.927540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d9eb3-44c4-41ee-9855-39e431028862 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:53.943147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:53.943800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:53.944304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:53.944780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] PATH_INFO: `/v3/projects/0b443d2af7944dd18b6325287ec99c46` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:53.947783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b443d2af7944dd18b6325287ec99c46)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:53.951122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:54.212905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b443d2af7944dd18b6325287ec99c46', 'request_id': 'req-6f9f76e3-09d6-4c42-9574-0a2452338930'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:54.230938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b443d2af7944dd18b6325287ec99c46', 'request_id': 'req-6f9f76e3-09d6-4c42-9574-0a2452338930'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:54.277169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b443d2af7944dd18b6325287ec99c46', 'request_id': 'req-6f9f76e3-09d6-4c42-9574-0a2452338930'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:54.514213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] The token cache is being invalidate because project 0b443d2af7944dd18b6325287ec99c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:54.520775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:54.522423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6f9f76e3-09d6-4c42-9574-0a2452338930 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:54.524609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1090/2199] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:53 2022] DELETE /identity/v3/projects/0b443d2af7944dd18b6325287ec99c46 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:54.530382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:54.531051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:54.531136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:54.594018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:54.595762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:54.781139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1110/2200] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:54.788935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:54.796754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:54.796754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:54.844186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:54.845482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:54.866160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1091/2201] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:54.876476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:54.892687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b38c35f-9db8-4154-a244-bcca281dcb41 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:54.899497 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:54.900356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:54.900887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:54.901509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:54.903040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:03:54.903515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:54.905352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:54.929919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1111/2202] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:03:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:03:54.948383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:54.961283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b6cea5-54e3-4dd1-a94d-a8a215687a4a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:54.972075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:54.972075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:54.972075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:54.972617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:54.974059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:54.978119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.064268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.066873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.082309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243dc73-206e-49a7-9563-d02961146d84 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.090836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.091611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.094996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.097002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] PATH_INFO: `/v3/users/a61748468b1245fc83f54b74f06e744e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.098891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a61748468b1245fc83f54b74f06e744e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.103058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1092/2203] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:03:54 2022] POST /identity/v3/projects => generated 373 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:03:55.104501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.107863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.226101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a433c9c7-572a-49af-9f60-49f9c1953dd8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.232721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.233712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.234312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.234683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.243952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.246804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.335093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a61748468b1245fc83f54b74f06e744e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.335093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a61748468b1245fc83f54b74f06e744e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.341935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a61748468b1245fc83f54b74f06e744e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.342670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a61748468b1245fc83f54b74f06e744e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.372448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1093/2204] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:03:55 2022] POST /identity/v3/users => generated 287 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:03:55.373753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.375953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.387200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d2fbcd5e-6b87-441d-84f5-fa2d0b6cfed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a61748468b1245fc83f54b74f06e744e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:55.397869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80421171-fd2a-43dc-85d1-99287143dae7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.405751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.406705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.407222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.408302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.409509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.411749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.431603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1094/2205] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:55.433848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.453457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c0aa-d62f-4037-a4f3-7f21e5f22ff4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.464214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.465180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.465637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.466066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.466923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.469433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.484282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1095/2206] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:03:55 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:03:55.488647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.503016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a641e0-a347-4028-9f9a-41f560e80158 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.513221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.521015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1112/2207] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:55 2022] DELETE /identity/v3/users/a61748468b1245fc83f54b74f06e744e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:55.522417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.523741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.524940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.525429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.525858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] PATH_INFO: `/v3/projects/fe528c21e8c94eb6af8d5f86757328b8/users/1729bb4d783f425085185c344dfe2ab8/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.531313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.547256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe528c21e8c94eb6af8d5f86757328b8, user_id=1729bb4d783f425085185c344dfe2ab8, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.551301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5320955c-8fb7-4b5f-8a3e-55cab663bec9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:55.565260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:55.566733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.674015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1096/2208] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:03:55 2022] PUT /identity/v3/projects/fe528c21e8c94eb6af8d5f86757328b8/users/1729bb4d783f425085185c344dfe2ab8/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:03:55.680779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.681853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.682858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.694015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1113/2209] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:55.703855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.709325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.718361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:55.720187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:55.858680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1097/2210] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:55.867654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bea237e-1fcb-473b-ba42-a477b0b131f9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.869531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:55.899734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:55.951144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859010a-bcd2-4ab1-92e2-277daca8463b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:55.956442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.957440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.957952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.958465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.972101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:55.972782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:55.981453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:55.981453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:55.981453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:55.981938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.002047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1098/2211] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:56.008358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:56.009829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:56.031403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1114/2212] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:56.362857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:57.175113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:57.177314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:57.204429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aa329f-1908-411c-975f-8600f36bcc13 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:57.213045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:57.214690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:57.216380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:57.216762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] PATH_INFO: `/v3/projects/71fc374a856a4043ba26051cba4ff20c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:57.219257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71fc374a856a4043ba26051cba4ff20c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:57.224395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:57.432013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:57.458875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71fc374a856a4043ba26051cba4ff20c', 'request_id': 'req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:57.467078 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71fc374a856a4043ba26051cba4ff20c', 'request_id': 'req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:57.475141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71fc374a856a4043ba26051cba4ff20c', 'request_id': 'req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:57.580376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] The token cache is being invalidate because project 71fc374a856a4043ba26051cba4ff20c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:57.580858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:57.584769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ef12cfaa-ca84-4ce8-8c4a-1bb96dd2dcd4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:57.587779 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1099/2213] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:57 2022] DELETE /identity/v3/projects/71fc374a856a4043ba26051cba4ff20c => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:57.596732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:57.628216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b88dc5e-80a6-4f93-8550-e73bb5b13af1 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:57.752853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:57.754846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:57.755606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:57.756461 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] PATH_INFO: `/v3/users/a6f59ef659f3482f85b7c30d3fcc73c3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:57.759911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6f59ef659f3482f85b7c30d3fcc73c3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:57.764767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:57.933357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:58.107322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6f59ef659f3482f85b7c30d3fcc73c3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:58.108258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6f59ef659f3482f85b7c30d3fcc73c3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:58.120504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6f59ef659f3482f85b7c30d3fcc73c3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:58.121548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6f59ef659f3482f85b7c30d3fcc73c3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:58.152599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6f59ef659f3482f85b7c30d3fcc73c3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:58.275651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1115/2214] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:03:57 2022] DELETE /identity/v3/users/a6f59ef659f3482f85b7c30d3fcc73c3 => generated 0 bytes in 682 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:58.512539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:58.516751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:58.517313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:58.596241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] MFA Rules not processed for user `5d8edf055ab9430ca2af83e70ad27f66`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:58.597599 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:58.742579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1100/2215] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:58 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:58.758044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:58.759818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:58.925239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbb500-8246-4064-a7f0-dc309bf9533d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:58.941691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:58.944814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:58.945404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:58.945867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:58.950214 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:59.112067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:59.121889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee266c6-0e86-41b5-bde2-b0eaea2cfbe3 service glance] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:59.146016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1116/2216] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:03:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:03:59.340890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:59.344019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:59.541482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3e4c19-438d-4545-ae63-61c902dd98ac None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:03:59.548722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:03:59.550192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:59.550711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:59.551136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] PATH_INFO: `/v3/projects/1a4eb29cbc0d45038b1686022ba628a7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:59.552729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a4eb29cbc0d45038b1686022ba628a7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:03:59.555010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:03:59.600863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:59.600863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:59.600863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:03:59.648875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:03:59.650592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:59.786296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:03:59.788083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a4eb29cbc0d45038b1686022ba628a7', 'request_id': 'req-69100d75-715f-4a6a-bb49-71148970079c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:59.799279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a4eb29cbc0d45038b1686022ba628a7', 'request_id': 'req-69100d75-715f-4a6a-bb49-71148970079c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:59.812984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a4eb29cbc0d45038b1686022ba628a7', 'request_id': 'req-69100d75-715f-4a6a-bb49-71148970079c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:59.815386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1117/2217] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:03:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:03:59.889237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:03:59.896765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:03:59.972598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] The token cache is being invalidate because project 1a4eb29cbc0d45038b1686022ba628a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:03:59.973120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:59.974030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-69100d75-715f-4a6a-bb49-71148970079c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:03:59.976031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1101/2218] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:03:59 2022] DELETE /identity/v3/projects/1a4eb29cbc0d45038b1686022ba628a7 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:03:59.984639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d848e5-bb61-465a-9a0f-10d31989eee9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:03:59.999028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d848e5-bb61-465a-9a0f-10d31989eee9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:03:59.999028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d848e5-bb61-465a-9a0f-10d31989eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.020336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-04d848e5-bb61-465a-9a0f-10d31989eee9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:00.022018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-04d848e5-bb61-465a-9a0f-10d31989eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:00.028667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005fa64b-f96a-4944-9983-ec7547a50116 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:00.178960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1102/2219] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:03:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:00.181200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0745060-897c-4b87-854f-18beae60d89f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:00.181528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0745060-897c-4b87-854f-18beae60d89f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:00.181805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0745060-897c-4b87-854f-18beae60d89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.188582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:00.188582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:00.188582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:00.188582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.190363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:00.228556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b0745060-897c-4b87-854f-18beae60d89f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:00.230438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b0745060-897c-4b87-854f-18beae60d89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:00.268748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1103/2220] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:00.271060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0745060-897c-4b87-854f-18beae60d89f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:00.295631 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0745060-897c-4b87-854f-18beae60d89f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:00.300998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:00.307220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:00.309588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:00.310649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:00.311005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:00.312461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.313403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:00.313718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:00.315372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:00.320375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1118/2221] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:03:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:04:00.341200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1104/2222] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:00.348295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:00.366493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf147e-61a9-40a0-a50e-9f4a785ca0c1 service swift] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:00.382370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:00.383007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:00.383307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:00.383594 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.384242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:00.386227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:00.487467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:00.573723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1119/2223] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:00 2022] POST /identity/v3/projects => generated 381 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:00.580349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:00.631126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc7b4b4-2920-4242-bcf4-c2a8be57e8e4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:00.849955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:00.854057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:00.854057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:00.854057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:00.854695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:00.858185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:01.054261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1105/2224] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:00 2022] POST /identity/v3/users => generated 291 bytes in 474 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:01.055674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:01.071812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92526cbb-3e77-4f64-8247-432b7511e862 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:01.080366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:01.081490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:01.082370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:01.083336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:01.084319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:01.086221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:01.099850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1120/2225] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:01.107766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:01.140051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5708e6-d3bf-4ad0-a1ba-159381426f88 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:01.149712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:01.151292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:01.151692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:01.152625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:01.153433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:01.155415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355ff638-eeaa-4f79-968e-026b3d46b5fd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:01.199685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1106/2226] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:01 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:01.200946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:01.213320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec9cd07-79c0-4c13-a642-d2ec35240a0f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:01.240722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:01.242585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:01.243422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:01.244230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] PATH_INFO: `/v3/projects/7f928cd84f2b4899a969686a1e0e8513/users/2b1d8ed4f22c4ab790961457a8c9db84/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:01.276506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f928cd84f2b4899a969686a1e0e8513, user_id=2b1d8ed4f22c4ab790961457a8c9db84, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:01.284993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:01.386374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1121/2227] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:01 2022] PUT /identity/v3/projects/7f928cd84f2b4899a969686a1e0e8513/users/2b1d8ed4f22c4ab790961457a8c9db84/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:01.400049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:01.400525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:01.400876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:01.468708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] MFA Rules not processed for user `2b1d8ed4f22c4ab790961457a8c9db84`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:01.470092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:01.700504 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1107/2228] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:01.705177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:01.831130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e4657-3e3a-465d-97bf-616f8fb39fb1 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:01.837655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:01.838290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:01.838579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:01.838854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:01.839584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:01.844380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:01.994706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1122/2229] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:01 2022] POST /identity/v3/projects => generated 379 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:01.997769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:02.045708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7f1766-6305-4a27-ace0-dfe8122eb591 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:02.281571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:02.284946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:02.287338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:02.287809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:02.288658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:02.300345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:02.501166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1108/2230] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:01 2022] POST /identity/v3/users => generated 296 bytes in 504 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:02.508045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:02.533341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83691ee3-7aeb-4c12-ae36-a467adc930c0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:02.571566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:02.575444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:02.576075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:02.576601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:02.578706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:02.584628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:02.620129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1123/2231] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:02.620543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:02.632827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5adfb5-4be3-4aab-b168-5c36e63494ac admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:02.656094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:02.660741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:02.660741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:02.660741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] PATH_INFO: `/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d/users/19bda6e03b1e4dfc9c950fbaad1a7270/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:02.686358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb56560caa84a8fb7bd3cb7ff52fa3d, user_id=19bda6e03b1e4dfc9c950fbaad1a7270, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:02.693323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:02.803132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1109/2232] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:02 2022] PUT /identity/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d/users/19bda6e03b1e4dfc9c950fbaad1a7270/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:02.810335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:02.834509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8148b0-09a2-4f54-8e3f-2bf95a6c911b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:02.980450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:02.981310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:02.981701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:02.982139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:02.983751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:02.985965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf17a0-ed52-46a2-bb14-ff566c7cb800 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:03.001598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1124/2233] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:02 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:03.002875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:03.014787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df27fd1e-9ae3-470e-8392-f99fdad7e9c7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:03.029282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:03.031021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:03.031528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:03.032053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] PATH_INFO: `/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d/users/19bda6e03b1e4dfc9c950fbaad1a7270/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:03.045994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb56560caa84a8fb7bd3cb7ff52fa3d, user_id=19bda6e03b1e4dfc9c950fbaad1a7270, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:03.049316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:03.116903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1110/2234] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:03 2022] PUT /identity/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d/users/19bda6e03b1e4dfc9c950fbaad1a7270/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:03.118079 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:03.118384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:03.118656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:03.177256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:03.192595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] MFA Rules not processed for user `19bda6e03b1e4dfc9c950fbaad1a7270`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:03.193973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:03.426960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1125/2235] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:03 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:04:03.439043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:03.442746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:03.606180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815c6c2-54f2-411f-b986-6faaeff2f381 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:03.615501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:03.616191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:03.616475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:03.616768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:03.641356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:03.643949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a8be10-80de-49e4-8cfb-32a03b3c567e service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:03.664372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:03.674594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1111/2236] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:03 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:04.537532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:04.537532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:04.704003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98845ace-79a7-49f7-891a-c49bdd7d30a3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:04.715280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:04.716121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:04.716753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:04.717278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] PATH_INFO: `/v3/users/491fb54f385e4f79bbaf6f9895d69997` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:04.719866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=491fb54f385e4f79bbaf6f9895d69997)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:04.723414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:05.220472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '491fb54f385e4f79bbaf6f9895d69997'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:05.221696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '491fb54f385e4f79bbaf6f9895d69997'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:05.259964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '491fb54f385e4f79bbaf6f9895d69997'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:05.261794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '491fb54f385e4f79bbaf6f9895d69997'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:05.323714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '491fb54f385e4f79bbaf6f9895d69997'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:05.412288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1126/2237] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:04 2022] DELETE /identity/v3/users/491fb54f385e4f79bbaf6f9895d69997 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:05.416116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:05.417346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:05.417817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:05.471192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:05.476678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:05.556861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:05.559859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:05.571351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:05.681958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:05.710365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1112/2238] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:05.711448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:05.718869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:05.801306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85302d-3abb-41c8-93a2-168c7d7fa305 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:05.813931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19bda6e03b1e4dfc9c950fbaad1a7270', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:05.814553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:05.814836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:05.815117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:05.815788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:05.816094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:05.818160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:05.853195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1127/2239] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:05.854313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:05.876577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1dd23-a4ed-43fe-a4a2-7ac785adfde0 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:05.890107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19bda6e03b1e4dfc9c950fbaad1a7270', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:05.890742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:05.891031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:05.891383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:05.892040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:05.894238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:05.923156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326e4e39-38fb-43ee-9d60-1c8d357dc9d3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:05.938491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:05.940097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:05.940645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:05.940924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:05.960203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:05.961914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:05.978322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1113/2240] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:06.062828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1128/2241] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:05 2022] POST /identity/v3/projects => generated 354 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:06.065098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:06.086615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46000d9-b07c-4b92-a987-50b6d6bbfcbe service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:06.318155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19bda6e03b1e4dfc9c950fbaad1a7270', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:06.319660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:06.320324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:06.325573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:06.325573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:06.326755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:06.329933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e664f997-5b16-4d9d-aea7-ac345bc6b57b tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:06.398703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1114/2242] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:06.496701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:06.516859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c208df-ca1a-42e5-8c9f-704724c64e5c tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:06.522977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '19bda6e03b1e4dfc9c950fbaad1a7270', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:06.524403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:06.525151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:06.525615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] PATH_INFO: `/v3/projects/cac6f76eb74345408f407a5a9f166920` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:06.545213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorizing `identity:delete_project(project_id=cac6f76eb74345408f407a5a9f166920)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:06.553997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:06.976346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cac6f76eb74345408f407a5a9f166920', 'request_id': 'req-0ff048cb-823a-43b7-b488-a61578bf1067'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.006379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cac6f76eb74345408f407a5a9f166920', 'request_id': 'req-0ff048cb-823a-43b7-b488-a61578bf1067'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.041765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cac6f76eb74345408f407a5a9f166920', 'request_id': 'req-0ff048cb-823a-43b7-b488-a61578bf1067'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.458795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:07.459753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:07.460134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:07.517443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] MFA Rules not processed for user `1729bb4d783f425085185c344dfe2ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:07.520567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.593164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] The token cache is being invalidate because project cac6f76eb74345408f407a5a9f166920 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:07.596048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.605908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:07.605908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1129/2243] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:06 2022] DELETE /identity/v3/projects/cac6f76eb74345408f407a5a9f166920 => generated 0 bytes in 1108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:07.624050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:07.628992 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.673880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1115/2244] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:07.677426 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:07.690844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:07.789752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff048cb-823a-43b7-b488-a61578bf1067 tempest-VolumeQuotaClassesTest-308560553 tempest-VolumeQuotaClassesTest-308560553-project-admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:07.814191 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:07.816014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:07.817272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:07.817745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] PATH_INFO: `/v3/projects/86f4ebeaa21949a1b2736cba21a26d1c` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:07.824855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=86f4ebeaa21949a1b2736cba21a26d1c)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:07.828668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:07.873427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327dafc-b398-4410-9279-3413b7e9f9f7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:07.879365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:07.884638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:07.885585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:07.886025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:07.887625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:07.889977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.050279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86f4ebeaa21949a1b2736cba21a26d1c', 'request_id': 'req-60c1b64d-fb84-4e81-b672-897594775166'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:08.059376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86f4ebeaa21949a1b2736cba21a26d1c', 'request_id': 'req-60c1b64d-fb84-4e81-b672-897594775166'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:08.071327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86f4ebeaa21949a1b2736cba21a26d1c', 'request_id': 'req-60c1b64d-fb84-4e81-b672-897594775166'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:08.078005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1116/2245] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:07 2022] POST /identity/v3/projects => generated 373 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:08.082064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.130880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:08.198575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] The token cache is being invalidate because project 86f4ebeaa21949a1b2736cba21a26d1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:08.199329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:08.200244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-60c1b64d-fb84-4e81-b672-897594775166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:08.204231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1130/2246] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:07 2022] DELETE /identity/v3/projects/86f4ebeaa21949a1b2736cba21a26d1c => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:08.223186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c181e-15cb-4ada-9acd-ee3ccf70b85c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.224212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c181e-15cb-4ada-9acd-ee3ccf70b85c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.225448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4c181e-15cb-4ada-9acd-ee3ccf70b85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.263628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-bc4c181e-15cb-4ada-9acd-ee3ccf70b85c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:08.266863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bc4c181e-15cb-4ada-9acd-ee3ccf70b85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:08.267081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44b1ab4-173c-4c29-81e1-17764af6610e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.280875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.281505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.282541 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1131/2247] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:08.285898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.286260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.287450 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.293902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.299764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.300046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.300503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.359064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:08.359064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:08.416771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1132/2248] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:08.418213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.446067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd71651-81eb-475c-bfa9-f45dbdd3f2a9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.465947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.469152 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:08.470527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.472490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.495020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1117/2249] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:08 2022] POST /identity/v3/users => generated 293 bytes in 413 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:04:08.498215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.504305 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:08.527791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a391eab-9565-467d-b4f5-1426bb2b29b3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.532041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1133/2250] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:08.533031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.543356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.544568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.546421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.547570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.553659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.557214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.565420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5007f3d-7f1c-4d4a-83c8-7a2a6f499bc2 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.579482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.580473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.580917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.581342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.582161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.585773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.629266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1118/2251] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:08 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:08.632482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.683064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6bef1-6c6a-404d-9a68-78a8efd8116e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.695190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.696907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.697823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.698168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] PATH_INFO: `/v3/projects/586baaa868fa4165acc453adf529d01e/users/82346818adfb42f2a1dfbbfc695e9e3f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.720721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586baaa868fa4165acc453adf529d01e, user_id=82346818adfb42f2a1dfbbfc695e9e3f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.725218 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:08.734448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1134/2252] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:08 2022] POST /identity/v3/projects => generated 379 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:08.736223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.738230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:08.833683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1119/2253] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:08 2022] PUT /identity/v3/projects/586baaa868fa4165acc453adf529d01e/users/82346818adfb42f2a1dfbbfc695e9e3f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:08.836741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:08.888909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b9d4d3-4b3e-470d-be32-4d5ee182e594 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.901849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:08.976244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a93fd-a832-4648-8c9d-012a0f08bed6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:08.982554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:08.983565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:08.984165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:08.984558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:08.986154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:08.989464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.006970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.008251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.008544 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.008859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] PATH_INFO: `/v3/users/2b1d8ed4f22c4ab790961457a8c9db84` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.010811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b1d8ed4f22c4ab790961457a8c9db84)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.014666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.084755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1120/2254] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:08 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:09.085756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.099866 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45874083-e667-44fd-9409-4d30b5a288b1 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.105681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.106325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.111554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.111554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.111554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.111554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.124472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1121/2255] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:09.125500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.146122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370ddc0-6e14-4a32-a70a-5d5d59e6b1eb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.154360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.155145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.155505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.155881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.156691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.158368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.173703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1122/2256] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:09.182062 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.207281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cad1a5-25b3-4f61-9a90-8373c7c74df0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.215371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.216129 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.216447 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.216725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] PATH_INFO: `/v3/projects/586baaa868fa4165acc453adf529d01e/users/82346818adfb42f2a1dfbbfc695e9e3f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.219190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=586baaa868fa4165acc453adf529d01e, user_id=82346818adfb42f2a1dfbbfc695e9e3f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.222122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.235180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1123/2257] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:09 2022] PUT /identity/v3/projects/586baaa868fa4165acc453adf529d01e/users/82346818adfb42f2a1dfbbfc695e9e3f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:09.236045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.288970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b1d8ed4f22c4ab790961457a8c9db84'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:09.290661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b1d8ed4f22c4ab790961457a8c9db84'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:09.317886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b1d8ed4f22c4ab790961457a8c9db84'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:09.321082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b1d8ed4f22c4ab790961457a8c9db84'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:09.353737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-ab2e9ad5-fbb6-45fb-a338-c993979cea64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b1d8ed4f22c4ab790961457a8c9db84'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:09.383156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:09.384232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba61944f-b569-4b38-ad01-593b7d3e5d28 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.395391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.396710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.397219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.397703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.401054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.403189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.418420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1124/2258] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:09 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:09.423644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.430880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:09.459006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1135/2259] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:08 2022] DELETE /identity/v3/users/2b1d8ed4f22c4ab790961457a8c9db84 => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:09.466629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.467114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.467586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.480609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa27728-ec8c-42da-aeaf-d67907101274 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.523494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:09.544106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:09.545482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:09.665729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1136/2260] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:09.685848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:09.687810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:09.691764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.707851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.708456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.708747 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] PATH_INFO: `/v3/projects/ae31829bffd7483fbab3dc87635aea58/users/26aa79753d6e4e49ad66cc32ba786fe8/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.764357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae31829bffd7483fbab3dc87635aea58, user_id=26aa79753d6e4e49ad66cc32ba786fe8, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.770509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.815930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83b010-890a-411f-9807-d88ba1c35786 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:09.820668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:09.823174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.823174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.823174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.836990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:09.839301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9f85e-3bd3-4319-8fa2-e6791870af09 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:09.853999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1137/2261] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:04:09.936578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:09.937064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:09.937540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:09.943135 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1125/2262] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:09 2022] PUT /identity/v3/projects/ae31829bffd7483fbab3dc87635aea58/users/26aa79753d6e4e49ad66cc32ba786fe8/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:09.987318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:09.989763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:10.160120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1138/2263] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:10.172525 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:10.174492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:10.324526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeedae4-1823-4d28-97bc-f013c2b77367 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:10.351054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:10.351755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:10.352041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:10.352343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:10.385294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:10.387320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:10.438714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1126/2264] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:04:10.883593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:10.887187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:10.954270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1816fe-1491-472e-a23b-d80224f5b273 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:10.990532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:10.996722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:10.997212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:11.000208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] PATH_INFO: `/v3/projects/7f928cd84f2b4899a969686a1e0e8513` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:11.013531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f928cd84f2b4899a969686a1e0e8513)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:11.017390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:11.080157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:11.208066 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:11.284963 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f928cd84f2b4899a969686a1e0e8513', 'request_id': 'req-bfcb9fe8-b385-4b67-add5-de08c0223584'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:11.299207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f928cd84f2b4899a969686a1e0e8513', 'request_id': 'req-bfcb9fe8-b385-4b67-add5-de08c0223584'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:11.319833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f928cd84f2b4899a969686a1e0e8513', 'request_id': 'req-bfcb9fe8-b385-4b67-add5-de08c0223584'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:11.565603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] The token cache is being invalidate because project 7f928cd84f2b4899a969686a1e0e8513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:11.566151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:11.567268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:11.572082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1139/2265] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:10 2022] DELETE /identity/v3/projects/7f928cd84f2b4899a969686a1e0e8513 => generated 0 bytes in 686 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:11.575586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:11.592594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2a74a5-e8e1-419f-86f5-191193a36a7f service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:11.776621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:11.777321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:11.777626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:11.777902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] PATH_INFO: `/v3/users/19bda6e03b1e4dfc9c950fbaad1a7270` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:11.780745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=19bda6e03b1e4dfc9c950fbaad1a7270)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:11.783342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:12.116255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19bda6e03b1e4dfc9c950fbaad1a7270'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:12.122055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19bda6e03b1e4dfc9c950fbaad1a7270'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:12.137222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19bda6e03b1e4dfc9c950fbaad1a7270'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:12.161212 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19bda6e03b1e4dfc9c950fbaad1a7270'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:12.166301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-c6829e56-8246-4e35-9edc-732c164da1fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19bda6e03b1e4dfc9c950fbaad1a7270'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:12.283157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1127/2266] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:11 2022] DELETE /identity/v3/users/19bda6e03b1e4dfc9c950fbaad1a7270 => generated 0 bytes in 710 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:14.145151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:14.146719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:14.168969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcb9fe8-b385-4b67-add5-de08c0223584 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:14.355409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:14.356075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:14.356394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:14.356684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] PATH_INFO: `/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:14.363674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb56560caa84a8fb7bd3cb7ff52fa3d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:14.367278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:14.538698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:14.656609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'request_id': 'req-44a97958-0bd4-4006-9741-7186ab6a882d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:14.698090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'request_id': 'req-44a97958-0bd4-4006-9741-7186ab6a882d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:14.733163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb56560caa84a8fb7bd3cb7ff52fa3d', 'request_id': 'req-44a97958-0bd4-4006-9741-7186ab6a882d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:14.961483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:14.971525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] The token cache is being invalidate because project 6fb56560caa84a8fb7bd3cb7ff52fa3d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:14.971899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:14.973256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-44a97958-0bd4-4006-9741-7186ab6a882d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:14.975675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1140/2267] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:14 2022] DELETE /identity/v3/projects/6fb56560caa84a8fb7bd3cb7ff52fa3d => generated 0 bytes in 831 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:14.983009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:14.983227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:14.983479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.043844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:15.049306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:15.244185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1128/2268] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:15.247681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:15.251968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:15.251968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.323393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:15.324952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:15.345882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1141/2269] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:15.350183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:15.405443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e3bd7-709f-46c9-be7e-eb35ccf5488e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:15.430261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:15.431166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:15.431633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:15.432170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.434550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:15.435064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:15.438370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:15.477420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1129/2270] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:15.482525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:15.496883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e5cd6f-bf43-48c9-951e-02c0be3c2e25 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:15.513420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:15.514054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:15.514333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:15.514611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.515456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:15.517431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:15.645709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1142/2271] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:15 2022] POST /identity/v3/projects => generated 381 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:15.648084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:15.668609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691fea60-1b06-4995-8b8b-3f41b177bd1c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:15.881512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:15.882614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:15.884708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:15.887817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:15.889139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:15.893977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:16.042623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1130/2272] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:15 2022] POST /identity/v3/users => generated 291 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:16.047334 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:16.070837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66992358-67b2-4aeb-b527-599cc233a3b5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:16.083409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:16.088098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:16.090745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:16.091113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:16.091826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:16.099622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:16.137415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1143/2273] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:16.140250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:16.151552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c233f2-cd73-404e-8d2f-fee2c9b44724 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:16.166053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:16.172676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:16.172676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:16.172676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:16.172676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:16.172676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df15c94-c0ce-4124-a1c7-4f9534165ecc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:16.197209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1131/2274] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:16.202457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:16.220452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b64a44-c4b7-4dd5-ac32-1f0ba0fe362f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:16.234473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:16.235874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:16.236687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:16.237192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] PATH_INFO: `/v3/projects/dedf0b589fd34e4ea4197d328d6ba51a/users/bee9d0df44fc4251b74b5a65b7714c36/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:16.312087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=dedf0b589fd34e4ea4197d328d6ba51a, user_id=bee9d0df44fc4251b74b5a65b7714c36, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:16.329748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:16.622646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1144/2275] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:16 2022] PUT /identity/v3/projects/dedf0b589fd34e4ea4197d328d6ba51a/users/bee9d0df44fc4251b74b5a65b7714c36/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:16.622838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:16.622838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:16.623169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:16.758348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] MFA Rules not processed for user `bee9d0df44fc4251b74b5a65b7714c36`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:16.781028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:17.087661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1132/2276] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:16 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 467 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:17.092443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:17.229854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15893e55-5b71-4e5a-8b6d-9faa37d19e0e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:17.244523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:17.245414 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:17.245770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:17.246089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:17.246703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:17.250292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:17.419364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1145/2277] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:17 2022] POST /identity/v3/projects => generated 383 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:17.423204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:17.450017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0385ae96-7b88-41e9-9a40-7b7da9b7681e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:17.593239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:17.594332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:17.594746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:17.595154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:17.595962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:17.603163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:17.791410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1133/2278] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:17 2022] POST /identity/v3/users => generated 298 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:17.794684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:17.809846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72213d2f-74b5-4984-80fc-1d1504b7e58b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:17.837239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:17.837239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:17.837239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:17.837239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:17.837711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:17.837711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:17.875146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1146/2279] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:17 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:17.879780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:17.894730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fc58f8-a290-4f70-a7ea-3c0f2059def0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:17.916335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:17.917325 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:17.917654 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:17.918074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] PATH_INFO: `/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc/users/eb5692de30e7407ba497dfe65a3386ef/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:17.950579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:17.976727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=567a0e0ea70d4b2f98136c6e99e303fc, user_id=eb5692de30e7407ba497dfe65a3386ef, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:17.979794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:18.124612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1134/2280] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:17 2022] PUT /identity/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc/users/eb5692de30e7407ba497dfe65a3386ef/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:18.129094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:18.146941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9018ba-d219-4785-8ef2-b6c42cc52b2f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:18.340586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:18.341226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:18.341510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:18.341790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:18.342763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:18.344433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ecdebb-dd81-4d4b-bc77-69e245268b55 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:18.383377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1147/2281] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:18.388046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:18.403977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c1797-b95e-40f4-86ee-3ec0077d8ce6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:18.407172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:18.414948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:18.415652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:18.418628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:18.420058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] PATH_INFO: `/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc/users/eb5692de30e7407ba497dfe65a3386ef/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:18.428661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] RBAC: Authorizing `identity:create_grant(project_id=567a0e0ea70d4b2f98136c6e99e303fc, user_id=eb5692de30e7407ba497dfe65a3386ef, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:18.431531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:18.487707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:18.488180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:18.488402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:18.496754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1135/2282] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:18 2022] PUT /identity/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc/users/eb5692de30e7407ba497dfe65a3386ef/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:18.531127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:18.534280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:18.564936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] MFA Rules not processed for user `eb5692de30e7407ba497dfe65a3386ef`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:18.566207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:18.700862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b37007e-4f5c-4c55-800c-65bc06dc5192 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:18.715285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:18.716334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:18.716774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:18.718232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] PATH_INFO: `/v3/users/b3161900d4434cef94776d7b2e27f261` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:18.720718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3161900d4434cef94776d7b2e27f261)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:18.731843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:18.749750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1148/2283] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:18 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:18.769210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:18.772483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:18.978647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e28958-7c4c-4488-9839-9f6219f1443c None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:18.988123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:18.988840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:18.990380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:18.990863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:19.021112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:19.024358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a57e05f-48ce-41b9-aa49-2a1671d4c15e service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:19.082005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1149/2284] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:19.114365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3161900d4434cef94776d7b2e27f261'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:19.115319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3161900d4434cef94776d7b2e27f261'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:19.144736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3161900d4434cef94776d7b2e27f261'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:19.180852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3161900d4434cef94776d7b2e27f261'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:19.181757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3161900d4434cef94776d7b2e27f261'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:19.298593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1136/2285] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:18 2022] DELETE /identity/v3/users/b3161900d4434cef94776d7b2e27f261 => generated 0 bytes in 768 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:04:19.303690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:19.309797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:19.309797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:19.353812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:19.355711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:19.433559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:19.435279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:19.590857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1150/2286] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:19.617378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:19.620634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:19.639004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6e49f9-2562-4f56-97b6-7eac6803f0c3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:19.656974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:19.660022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:19.660350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:19.664557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:19.677391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:19.769909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464cb212-e812-4319-b430-7a37dbdc328f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:19.785412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:19.786049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:19.786339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:19.786691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:19.822048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:19.824029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:19.861887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1151/2287] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:19.878483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:19.880286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ede4c-be58-48ca-8a79-83d47dd13cd1 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:19.903313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1137/2288] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:04:20.978378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:20.980985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:21.014037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24064c0-78f2-4483-8726-f9a1efb391f9 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:21.038070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:21.039816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:21.040506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:21.040970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:21.041591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] PATH_INFO: `/v3/users/bee9d0df44fc4251b74b5a65b7714c36` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:21.043608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bee9d0df44fc4251b74b5a65b7714c36)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:21.047644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:21.412095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bee9d0df44fc4251b74b5a65b7714c36'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:21.412979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bee9d0df44fc4251b74b5a65b7714c36'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:21.440930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bee9d0df44fc4251b74b5a65b7714c36'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:21.442421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bee9d0df44fc4251b74b5a65b7714c36'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:21.482424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bee9d0df44fc4251b74b5a65b7714c36'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:21.601699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:21.607096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1152/2289] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:20 2022] DELETE /identity/v3/users/bee9d0df44fc4251b74b5a65b7714c36 => generated 0 bytes in 629 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:21.612298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:21.612773 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:21.613118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:21.663784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:21.668472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:21.819169 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1138/2290] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:21.830612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:21.832448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:21.970468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da856e1d-4a85-45da-ae7b-77c694945dd5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:21.970468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:21.970468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:21.970468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:21.971127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:22.015062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:22.019004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:22.046105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1153/2291] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:22.612910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:22.615811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:22.641183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6f6f6-3cae-41d1-886b-3ce3f209e46d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:22.671173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:22.671173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:22.671173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:22.671173 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] PATH_INFO: `/v3/projects/9e36b71df75a4f69b93ee08575030e3c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:22.672809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e36b71df75a4f69b93ee08575030e3c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:22.675198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:23.052696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e36b71df75a4f69b93ee08575030e3c', 'request_id': 'req-d6bf1c31-035d-4930-b4b7-ae77222b4d82'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:23.064484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e36b71df75a4f69b93ee08575030e3c', 'request_id': 'req-d6bf1c31-035d-4930-b4b7-ae77222b4d82'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:23.095859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e36b71df75a4f69b93ee08575030e3c', 'request_id': 'req-d6bf1c31-035d-4930-b4b7-ae77222b4d82'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:23.250428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] The token cache is being invalidate because project 9e36b71df75a4f69b93ee08575030e3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:23.250428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:23.250428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d6bf1c31-035d-4930-b4b7-ae77222b4d82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:23.250428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1139/2292] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:22 2022] DELETE /identity/v3/projects/9e36b71df75a4f69b93ee08575030e3c => generated 0 bytes in 620 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:23.808516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:23.810174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:23.836896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77f2d7c-66c0-4301-83ef-de16746dac75 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:24.001626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:24.007597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:24.007597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:24.007597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] PATH_INFO: `/v3/projects/dedf0b589fd34e4ea4197d328d6ba51a` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:24.007597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=dedf0b589fd34e4ea4197d328d6ba51a)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:24.007597 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:24.645911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dedf0b589fd34e4ea4197d328d6ba51a', 'request_id': 'req-976678d2-a5af-4657-bb99-7d9a01cf2cdc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:24.654240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dedf0b589fd34e4ea4197d328d6ba51a', 'request_id': 'req-976678d2-a5af-4657-bb99-7d9a01cf2cdc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:24.667363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dedf0b589fd34e4ea4197d328d6ba51a', 'request_id': 'req-976678d2-a5af-4657-bb99-7d9a01cf2cdc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:24.726104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:24.727094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:24.727479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:24.775056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] The token cache is being invalidate because project dedf0b589fd34e4ea4197d328d6ba51a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:24.779195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:24.779195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:24.779195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1154/2293] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:23 2022] DELETE /identity/v3/projects/dedf0b589fd34e4ea4197d328d6ba51a => generated 0 bytes in 970 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:24.782321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:24.784557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:24.813034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] MFA Rules not processed for user `82346818adfb42f2a1dfbbfc695e9e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:24.815247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:25.001317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1140/2294] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:25.013185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:25.019500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:25.020428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976678d2-a5af-4657-bb99-7d9a01cf2cdc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:25.044493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:25.045339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:25.045754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:25.046189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] PATH_INFO: `/v3/users/eb5692de30e7407ba497dfe65a3386ef` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:25.052806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb5692de30e7407ba497dfe65a3386ef)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:25.072084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:25.163119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92a412b-df30-40e4-8d38-56aa00140162 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:25.173328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:25.174298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:25.174737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:25.175180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:25.177718 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:25.368901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:25.387052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc01862-e2a0-4e9b-bc45-77c26d8c3801 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:25.402036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1141/2295] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:25.403341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:25.403556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:25.403834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:25.468289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] MFA Rules not processed for user `26aa79753d6e4e49ad66cc32ba786fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:25.475121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:25.494631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb5692de30e7407ba497dfe65a3386ef'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:25.499626 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb5692de30e7407ba497dfe65a3386ef'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:25.531627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb5692de30e7407ba497dfe65a3386ef'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:25.532628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb5692de30e7407ba497dfe65a3386ef'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:25.564721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb5692de30e7407ba497dfe65a3386ef'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:25.623928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1142/2296] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:25.632582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:25.637363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:25.683396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d047eb3-23ed-40d6-96e2-92376638d152 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:25.691680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:25.692944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:25.693232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:25.693506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:25.694154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:25.696191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:25.707999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1155/2297] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:24 2022] DELETE /identity/v3/users/eb5692de30e7407ba497dfe65a3386ef => generated 0 bytes in 923 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:25.878959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1143/2298] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:25 2022] POST /identity/v3/projects => generated 377 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:25.883292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:26.049911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df686678-2217-4c6c-9591-c90887130a1f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:26.235518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:26.236215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:26.236610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:26.236935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:26.237615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:26.239671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:26.652899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1156/2299] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:25 2022] POST /identity/v3/users => generated 295 bytes in 770 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:26.655194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:26.670137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad6d2f3-c314-414d-8bbd-55a17a648d0d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:26.679729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:26.679729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:26.679729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:26.679729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:26.680506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:26.682048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:26.715988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1144/2300] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:26.722881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:26.736848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0290dc5a-b3b3-4c0d-a18e-c990b3618764 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:26.743142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:26.744252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:26.746675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:26.747350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] PATH_INFO: `/v3/projects/71512d420f5245c8976eadd51e306229/users/690c034eae2d4448a9b6de152728c0ca/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:26.772536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] RBAC: Authorizing `identity:create_grant(project_id=71512d420f5245c8976eadd51e306229, user_id=690c034eae2d4448a9b6de152728c0ca, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:26.775455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:26.908456 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1157/2301] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:26 2022] PUT /identity/v3/projects/71512d420f5245c8976eadd51e306229/users/690c034eae2d4448a9b6de152728c0ca/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:04:26.909782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:26.922579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824efc74-6990-4a57-97f0-a36fac0271a0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:27.026166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:27.027145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.027627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.028559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.029738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:27.032058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:27.037714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:27.040047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:27.065809 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1145/2302] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:26 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:27.068015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7354ff0b-cc85-496f-82c6-aa2576c6e24e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:27.068777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:27.076229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:27.084274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.084274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.084274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] PATH_INFO: `/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.084791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3677457-8659-4f04-ab58-1d96f6d4f5b9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:27.089156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] RBAC: Authorizing `identity:delete_project(project_id=567a0e0ea70d4b2f98136c6e99e303fc)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:27.090719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:27.091444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.093875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.094183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] PATH_INFO: `/v3/projects/71512d420f5245c8976eadd51e306229/users/690c034eae2d4448a9b6de152728c0ca/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.094706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:27.096699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71512d420f5245c8976eadd51e306229, user_id=690c034eae2d4448a9b6de152728c0ca, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:27.100027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd1a63f-d040-44b2-af43-9d81969c54d4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:27.123643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1146/2303] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:27 2022] PUT /identity/v3/projects/71512d420f5245c8976eadd51e306229/users/690c034eae2d4448a9b6de152728c0ca/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:04:27.291016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '567a0e0ea70d4b2f98136c6e99e303fc', 'request_id': 'req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:27.301774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '567a0e0ea70d4b2f98136c6e99e303fc', 'request_id': 'req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:27.311138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '567a0e0ea70d4b2f98136c6e99e303fc', 'request_id': 'req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:27.397392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bff9d-498a-4981-930a-33c528dfcad4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.397658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bff9d-498a-4981-930a-33c528dfcad4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.397911 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31bff9d-498a-4981-930a-33c528dfcad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.432097 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a31bff9d-498a-4981-930a-33c528dfcad4 None None] MFA Rules not processed for user `775ee674f5084d01b48d7b3ef20c7864`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:27.433918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a31bff9d-498a-4981-930a-33c528dfcad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:27.436757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] The token cache is being invalidate because project 567a0e0ea70d4b2f98136c6e99e303fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:27.437393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:27.437961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-22f67d22-7a9f-4c5d-bbfa-da2971c7c13d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:27.447465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1158/2304] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:27 2022] DELETE /identity/v3/projects/567a0e0ea70d4b2f98136c6e99e303fc => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:27.451927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.457063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.457063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.494232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:27.495621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:27.510325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:27.551670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1147/2305] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:27.673828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1159/2306] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:27.675365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.675692 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.675929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.702167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:27.703974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:27.722158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:27.731690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1148/2307] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:27.737574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61025ac2-8ef2-4332-9fe7-c62df795c1ad None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:27.743790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:27.744775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.745664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.745664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.747370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:27.747799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:27.749606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:27.769660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1160/2308] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:27.774862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:27.786927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef7ecd4-3654-4fd4-a995-66d1efba2a91 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:27.795152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:27.801376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:27.801376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:27.801376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:27.801376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:27.803876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0630ea18-1a85-4dcf-98e2-4d5cd6a20aca admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:27.951520 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:28.003867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1149/2309] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:27 2022] POST /identity/v3/projects => generated 371 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:04:28.009332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:28.030776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a758008d-196b-4f0d-ba0c-317c37fd5353 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:28.176473 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:28.178047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.178349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.178895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.180946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:28.182531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.182903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.183216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.185737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:28.228382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] MFA Rules not processed for user `9b0569d9de724d5fbe3f41cbc0687d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:28.239475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:28.344816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1161/2310] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:28 2022] POST /identity/v3/users => generated 286 bytes in 336 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:28.353663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:28.405632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1150/2311] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:28.405834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51123995-5395-461f-b89c-9e35a7ce2687 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:28.425533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:28.426658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.427165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.435666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.437296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:28.445028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:28.478402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1162/2312] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:04:28.480676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:28.524933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b938ded2-66ba-4270-be95-fa7f11b3c221 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:28.545440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:28.546296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.546745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.547251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.548346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:28.552684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd75ba2-a741-4795-b33c-69e45ba9253f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:28.596630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1151/2313] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:28 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:04:28.600173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:28.618068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e192ee3b-78e3-4ec0-aed1-447c0a35d12a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:28.623638 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:28.624574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.624955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.625337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] PATH_INFO: `/v3/projects/5fa9e35cde2047d29f3ae272866c2a9a/users/07fb21067e43488ba21a8f9d1c73a058/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.653307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fa9e35cde2047d29f3ae272866c2a9a, user_id=07fb21067e43488ba21a8f9d1c73a058, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:28.656469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:28.876889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa19f1-20da-4240-bee3-3636a1f968d7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:28.877388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa19f1-20da-4240-bee3-3636a1f968d7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:28.882349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfa19f1-20da-4240-bee3-3636a1f968d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:28.883164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1163/2314] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:28 2022] PUT /identity/v3/projects/5fa9e35cde2047d29f3ae272866c2a9a/users/07fb21067e43488ba21a8f9d1c73a058/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:28.934395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5bfa19f1-20da-4240-bee3-3636a1f968d7 None None] MFA Rules not processed for user `07fb21067e43488ba21a8f9d1c73a058`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:28.935879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5bfa19f1-20da-4240-bee3-3636a1f968d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:29.102436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1152/2315] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:29.143429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:29.150545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:29.293124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c15fa3-1cb4-487c-a2b6-cc231e77975b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:29.303933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:29.306881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:29.311026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:29.311026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:29.348651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:29.350578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eeb6fc0-d96a-41b7-8eda-33e9eff16b44 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:29.363257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1164/2316] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:29.740769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:30.413715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:30.880941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:32.218492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:32.218492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:32.218492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:32.269924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] MFA Rules not processed for user `1729bb4d783f425085185c344dfe2ab8`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:32.271487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:32.370587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1153/2317] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:33.780826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:34.310531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:36.226680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:36.538771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c55511-5eae-4f95-ad8d-1e63827360c8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:36.539054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c55511-5eae-4f95-ad8d-1e63827360c8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:36.539331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c55511-5eae-4f95-ad8d-1e63827360c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:36.575459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-59c55511-5eae-4f95-ad8d-1e63827360c8 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:36.579394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-59c55511-5eae-4f95-ad8d-1e63827360c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:36.657595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1165/2318] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:04:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:36.683700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:36.685274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:36.870002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f126d1fc-4e8e-4f7a-a5f6-9dfdc3e68c8b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:36.877077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:36.877719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:36.878002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:36.878280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:36.893870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:36.894255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:36.899276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466160c1-5102-458f-a138-bd1a376f4b9f service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:36.911471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1154/2319] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:04:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:04:39.464347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:39.744037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:41.167341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=86,keystone:INSERT=5,keystone:DELETE=9 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:41.240833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:42.564423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:42.565955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:42.566567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:42.606313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] MFA Rules not processed for user `a30b0734975d4e5a848be44aa7860807`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:42.607546 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:42.799494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1166/2320] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:43.801087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:43.801622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:43.801622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:43.820860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:43.895681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] MFA Rules not processed for user `690c034eae2d4448a9b6de152728c0ca`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:43.898114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:44.089436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1155/2321] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:44.218631 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:44.462999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:44.466563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:44.607826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537f12c-05fd-46bb-93ca-1c6afad8d5a3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:44.635791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:44.637428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:44.637714 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:44.637993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:44.648813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:44.734375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:44.737737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abad8625-61d0-4677-92db-b4f87870b7d7 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:44.765746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1167/2322] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:44 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:47.631604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:47.948053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:48.554952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:48.801235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ef0fa4-53f7-44cc-9f7c-0d9f448ceea0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:48.801235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:48.801235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:48.801235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:48.802420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] PATH_INFO: `/v3/users/07fb21067e43488ba21a8f9d1c73a058` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:48.802420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] RBAC: Authorizing `identity:delete_user(user_id=07fb21067e43488ba21a8f9d1c73a058)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:48.802420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:50.117822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '07fb21067e43488ba21a8f9d1c73a058'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:50.119296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '07fb21067e43488ba21a8f9d1c73a058'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:50.137136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '07fb21067e43488ba21a8f9d1c73a058'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:50.181505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '07fb21067e43488ba21a8f9d1c73a058'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:50.189661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '07fb21067e43488ba21a8f9d1c73a058'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:50.304813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1156/2323] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:04:48 2022] DELETE /identity/v3/users/07fb21067e43488ba21a8f9d1c73a058 => generated 0 bytes in 1750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:50.309503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:50.310051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:50.310051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:50.330550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:50.331833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:50.443676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1168/2324] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:50.456420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:50.457907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:50.563740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108276c7-4d43-4659-b5de-109759934626 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:50.574004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:50.575603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:50.576017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:50.576400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:50.590112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:50.594929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337fab80-4fb7-4c77-95ca-dc4fbf03cfd0 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:50.607710 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1157/2325] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:51.647395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:51.649369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:51.670469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd775ec7-f1cb-45d8-87fe-4e2220a2cfdb None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:51.695579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:51.696625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:51.696925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:51.697295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] PATH_INFO: `/v3/projects/5fa9e35cde2047d29f3ae272866c2a9a` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:51.698967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fa9e35cde2047d29f3ae272866c2a9a)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:51.703604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:51.880687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fa9e35cde2047d29f3ae272866c2a9a', 'request_id': 'req-902e0469-3582-4eff-af38-3fac951e0d1e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:51.888272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fa9e35cde2047d29f3ae272866c2a9a', 'request_id': 'req-902e0469-3582-4eff-af38-3fac951e0d1e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:51.894333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fa9e35cde2047d29f3ae272866c2a9a', 'request_id': 'req-902e0469-3582-4eff-af38-3fac951e0d1e'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:52.012050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] The token cache is being invalidate because project 5fa9e35cde2047d29f3ae272866c2a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:04:52.012390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:52.013013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-902e0469-3582-4eff-af38-3fac951e0d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:04:52.014710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1169/2326] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:04:51 2022] DELETE /identity/v3/projects/5fa9e35cde2047d29f3ae272866c2a9a => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:52.019713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.020528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.020528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.042597 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:52.044653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:52.110812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1158/2327] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:52.117636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.118271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.119927 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.151310 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:52.153167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:52.166982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1170/2328] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:52.170471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:52.184454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccf49e0-f5c1-4df0-883e-8b99b87c7e29 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:52.190983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:52.192253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.193136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.193665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.194388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:04:52.194803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:52.196633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:52.209643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1159/2329] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:04:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:04:52.214271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:52.232536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e438c394-cdaf-4ccf-af6c-0f3d2dc233a4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:52.251571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:52.252648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.253141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.255887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.257115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:52.260624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:52.286726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:52.548237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1171/2330] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:04:52 2022] POST /identity/v3/projects => generated 373 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:52.552431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:52.563955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b0af78-2e8c-47d2-8c1a-731c3fe9fa2a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:52.706691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:52.707547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.707927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.708482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.729086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:52.729086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:52.813331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:52.919664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1160/2331] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:04:52 2022] POST /identity/v3/users => generated 287 bytes in 368 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:04:52.921505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:52.932189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f425e9-5b35-4b0d-8ecc-cfe9770c8ea8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:52.948080 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:52.948744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.950017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.950460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.951294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:52.953132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:52.964126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1172/2332] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:04:52.968453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:52.992577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc0314-dfe7-4173-a4e5-269a18d30b82 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:52.992577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:52.992577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:52.992577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:52.993159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:52.994015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:52.996715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ea70b-4f09-4255-a651-30cfeba29d78 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:53.008922 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1161/2333] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:04:52 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:04:53.013609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:53.024037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e38955-1e78-456c-b7df-d6496c23f4e2 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:53.030493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:53.031175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:53.031523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:53.031877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] PATH_INFO: `/v3/projects/9a212684b7034dd2a02eff9cc2a3dbd8/users/187691a095d94a6591ff94eeef7c41c7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:53.057351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a212684b7034dd2a02eff9cc2a3dbd8, user_id=187691a095d94a6591ff94eeef7c41c7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:53.057351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:53.157231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1173/2334] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:04:53 2022] PUT /identity/v3/projects/9a212684b7034dd2a02eff9cc2a3dbd8/users/187691a095d94a6591ff94eeef7c41c7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:04:53.170139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:53.170925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:53.171411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:53.205835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] MFA Rules not processed for user `187691a095d94a6591ff94eeef7c41c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:53.207324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:53.283841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1162/2335] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:53.296922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:53.299784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:53.360836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b3472-ff09-427e-87af-e3582eeff305 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:53.364550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:53.365179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:53.365462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:53.365841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:53.375171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:53.395033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c0c034-a888-4906-bee4-0c9a530490d9 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:53.395033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1174/2336] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:53.965225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:04:53.970392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:54.114316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15243329-80a3-47e0-a108-ea95edfc1b72 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:04:54.121624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:04:54.122585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:54.123063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:54.123581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:54.125952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:54.142962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:04:54.146294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab5b956-f1b3-4e06-bffb-bd5273744095 service placement] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:04:54.174504 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1163/2337] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:04:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:04:54.798630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:55.482289 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:04:55.484481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:04:55.484481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:04:55.502941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] MFA Rules not processed for user `187691a095d94a6591ff94eeef7c41c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:04:55.504272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:04:55.511823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1175/2338] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:04:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:04:56.853964 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:57.746322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:04:59.178565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:05.841710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:09.949888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:13.849796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:13.851294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:13.851664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:13.875549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] MFA Rules not processed for user `26aa79753d6e4e49ad66cc32ba786fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:13.876842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:13.937984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1164/2339] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:05:13 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:18.712068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:18.713502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:18.774577 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efcf590-5a9e-4e4c-b41d-af8e758764c9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:18.778820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:18.780030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:18.780030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:18.780030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:18.781990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:18.872985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:18.874680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d37c26-a3ec-4223-8656-b58a7e64b368 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:18.889273 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1176/2340] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:23.930241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=25 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:27.207398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:27.208854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:27.224069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37638dd6-b51c-4207-b6fb-a154f6f31a61 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:27.227797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:27.228628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:27.228793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:27.229241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:27.231293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:27.306471 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:27.308761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:27.321264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1165/2341] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:27 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:05:28.880896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=50 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:29.206950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b524e0-8ae7-4891-8094-fde677423bd3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:29.207260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b524e0-8ae7-4891-8094-fde677423bd3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:29.207538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b524e0-8ae7-4891-8094-fde677423bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:29.229014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-52b524e0-8ae7-4891-8094-fde677423bd3 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:29.230245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-52b524e0-8ae7-4891-8094-fde677423bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:29.288095 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1177/2342] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:05:29 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:29.310056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:29.311603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:29.407663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb570653-c868-4c81-ad6c-3a1ed7d9cf4a service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:29.412953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:29.413612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:29.416819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:29.416819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:29.427188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:29.427188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:29.436160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1166/2343] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:05:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:05:30.477362 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2635d4-2261-498e-9089-5f3946384782 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:30.477938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2635d4-2261-498e-9089-5f3946384782 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:30.478494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2635d4-2261-498e-9089-5f3946384782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:30.509436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1f2635d4-2261-498e-9089-5f3946384782 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:30.510962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1f2635d4-2261-498e-9089-5f3946384782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:30.527548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1178/2344] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:05:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:30.545702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:30.559262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31852c64-0421-4d2d-900d-cb9c0857b02a service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:30.565213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:30.566992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:30.567284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:30.567629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:30.577893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:30.579564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:30.590591 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1167/2345] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:05:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:05:31.415905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790319-93f2-45d8-b2a9-113a5d0051ca None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:31.416264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790319-93f2-45d8-b2a9-113a5d0051ca None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:31.416627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2790319-93f2-45d8-b2a9-113a5d0051ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:31.440335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a2790319-93f2-45d8-b2a9-113a5d0051ca None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:31.442642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a2790319-93f2-45d8-b2a9-113a5d0051ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:31.450823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1179/2346] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:05:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:31.466029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:31.478275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9eb740-e79b-4f4d-8a86-f80dc9955a77 service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:31.484798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:31.485518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:31.485882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:31.486231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:31.497656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:31.499419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:31.509350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1168/2347] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:05:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:05:32.232706 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:32.233467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:32.233467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:32.257407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:32.258794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:32.267851 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1180/2348] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:05:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:32.281414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:32.293776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955e3108-21e1-4867-a4bd-f97f93d40a7e service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:32.299578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:32.300405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:32.300782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:32.301151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:32.311926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:32.314071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6ecd3b-bd43-44c5-a53f-28decde35560 service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:32.324835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1169/2349] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:05:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:05:35.515911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:35.606896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc48061d-5db8-4bcb-bab7-1d84bf0bf288 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:35.611935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:35.612680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:35.612991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:35.613381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] PATH_INFO: `/v3/users/187691a095d94a6591ff94eeef7c41c7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:35.614567 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=187691a095d94a6591ff94eeef7c41c7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:35.616566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:35.825193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '187691a095d94a6591ff94eeef7c41c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:35.825916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '187691a095d94a6591ff94eeef7c41c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:35.833858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '187691a095d94a6591ff94eeef7c41c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:35.834549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '187691a095d94a6591ff94eeef7c41c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:35.847571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '187691a095d94a6591ff94eeef7c41c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:35.935804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1181/2350] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:05:35 2022] DELETE /identity/v3/users/187691a095d94a6591ff94eeef7c41c7 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:35.941423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:35.942002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:35.942375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:35.975422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:05:35.977670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:36.035014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:36.057611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1170/2351] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:05:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:05:36.070361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:36.072057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:36.079622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:36.146305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67791278-507f-4cf7-8363-91966f307e98 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:36.151053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:36.151589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:36.151869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:36.152165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:36.162718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:36.164227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:36.175730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1182/2352] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:36.454840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:36.456553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:36.473566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6f28c7-fcc4-4718-ba58-54af6842d656 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:36.484382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:36.485472 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:36.485970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:36.486448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] PATH_INFO: `/v3/projects/9a212684b7034dd2a02eff9cc2a3dbd8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:36.488672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a212684b7034dd2a02eff9cc2a3dbd8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:36.492476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:36.622986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a212684b7034dd2a02eff9cc2a3dbd8', 'request_id': 'req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:36.630757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a212684b7034dd2a02eff9cc2a3dbd8', 'request_id': 'req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:36.640722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a212684b7034dd2a02eff9cc2a3dbd8', 'request_id': 'req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:36.807563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] The token cache is being invalidate because project 9a212684b7034dd2a02eff9cc2a3dbd8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:05:36.807908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:36.809306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3b50ad6e-fdd2-4f75-aa8b-7c46da3ca751 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:05:36.811320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1171/2353] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:05:36 2022] DELETE /identity/v3/projects/9a212684b7034dd2a02eff9cc2a3dbd8 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:05:44.792179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:05:44.793712 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:44.806141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e006c5-b9f6-49f0-94f4-3628b9a76957 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:05:44.858312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:05:44.858823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:05:44.859099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:05:44.859368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:05:44.861089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:05:44.928678 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:05:44.930123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:05:44.940202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1183/2354] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:05:44 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:05:46.803283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=34,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:05:54.935677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=74 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:17.428934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83473-0c80-4d6f-b2c6-6fae49eff0d5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:17.429470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83473-0c80-4d6f-b2c6-6fae49eff0d5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:17.429556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b83473-0c80-4d6f-b2c6-6fae49eff0d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:17.462119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-56b83473-0c80-4d6f-b2c6-6fae49eff0d5 None None] MFA Rules not processed for user `9b0569d9de724d5fbe3f41cbc0687d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:17.463413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-56b83473-0c80-4d6f-b2c6-6fae49eff0d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:17.536522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1172/2355] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:06:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:27.529536 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=25 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:38.541221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:38.544463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:38.555076 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb30921-73a1-475d-8494-cd817b569a2c service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:38.559686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:38.559686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:38.560056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:38.560290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:38.562239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:38.641907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:38.643353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:38.657395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1184/2356] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:42.373489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:42.374024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:42.374024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:42.393598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] MFA Rules not processed for user `a30b0734975d4e5a848be44aa7860807`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:06:42.394851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:42.453146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1173/2357] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:06:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:06:44.063451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:44.072462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99748406-5de0-4f14-bca7-6f8b39ab99db service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:44.076665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:44.077257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:44.077539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:44.077812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:44.079549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:44.093916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:44.095325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:44.106557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1185/2358] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:45.243777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:06:45.245302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:45.261071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39ecb98-d115-4e29-a636-af2160817a87 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:06:45.266570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:06:45.267168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:06:45.267448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:06:45.267725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:06:45.269456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:06:45.281834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:06:45.283410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e6fd8f-2b7d-4dfc-8787-474ec5e34331 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:06:45.294382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1174/2359] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:06:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:06:54.101384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=40 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:06:55.288713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=31 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:02.045646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:02.047073 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:02.103914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317b358-e352-48c3-bc42-eefa2721bdd0 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:02.108170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:02.108775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:02.109053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:02.109323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:02.111201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:02.172371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:02.173703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:02.183266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1186/2360] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:12.157428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:12.157860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:12.157860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:12.178622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=48 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:12.179432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] MFA Rules not processed for user `c18dc6b5898c4e1aae71a028b4840bf0`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:12.180829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:12.187909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1175/2361] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:07:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:12.719239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:12.729457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241f3710-0339-4060-b3dc-2361c2802970 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:12.733685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:12.734434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:12.734720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:12.734997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:12.736979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:12.748012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:12.749523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:12.762002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1187/2362] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:12.971032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:12.972726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:12.995954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486992b-945f-4f08-a2a9-a299fee125a9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:13.004027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:13.009239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:13.009888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:13.010322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:13.013050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:13.031067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:13.036978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:13.056542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1176/2363] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:12 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:13.318130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:13.321905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:13.454186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802a531d-a59e-4766-8d65-fbfcf1d52ea5 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:13.466290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:13.467356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:13.467881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:13.468441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:13.471747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:13.497341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:13.499162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:13.512465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1188/2364] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:18.598819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:18.600563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:18.621636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ce7235-7f52-4dfe-a7c5-a090f3b9fe16 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:18.641080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:18.641839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:18.642644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:18.643125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:18.645713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:18.658120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:18.659984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2138343-14a5-4f2a-ba04-72a8d55f6585 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:18.673802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1177/2365] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:18.750960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:18.787144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ad35a6-4c56-4c39-b2c8-549f03568376 service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:18.799640 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:18.801961 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:18.802689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:18.803450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:18.807397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:18.832989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:18.834651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:18.848463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1189/2366] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:07:24.299375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:24.299858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:24.300315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:24.335277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:24.337108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:24.403216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1178/2367] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:07:24 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:24.557162 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:24.558604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:24.637664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb75fa1b-8503-4ef9-a44c-e4bb778b2c1a service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:24.643498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:24.644114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:24.644419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:24.644687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:24.654984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:24.657643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe8a71b-c3bd-40b9-bfce-fb2022df6c10 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:24.666141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1190/2368] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:07:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:07:27.269197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:27.284937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd8624-5d6c-4654-b911-43b8b56e6ae7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:27.292238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:27.293136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:27.293522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:27.294388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:27.297528 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:27.320463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:27.323361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f54a81d-8a19-43be-83b9-4f135a9b5aa4 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:27.340023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1179/2369] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:30.241861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cc54-8182-448d-a52c-9c3da107fe76 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.243003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cc54-8182-448d-a52c-9c3da107fe76 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.243003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f4cc54-8182-448d-a52c-9c3da107fe76 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.245041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1191/2370] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:07:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:07:30.279783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998844f-8e68-4059-bafe-cc0ed45ed51b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.280673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998844f-8e68-4059-bafe-cc0ed45ed51b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.280673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998844f-8e68-4059-bafe-cc0ed45ed51b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.310254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b998844f-8e68-4059-bafe-cc0ed45ed51b None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:30.311665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b998844f-8e68-4059-bafe-cc0ed45ed51b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:30.321521 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1180/2371] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:30.333585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6eaa-0d39-4c73-b2dd-04985bad466a None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.333969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6eaa-0d39-4c73-b2dd-04985bad466a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.334839 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97a6eaa-0d39-4c73-b2dd-04985bad466a None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.338243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1192/2372] 10.222.0.97 () {54 vars in 961 bytes} [Thu Sep 8 14:07:30 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:07:30.346444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.346962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.350552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.385679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] MFA Rules not processed for user `59c91fec50c6477b8f7b7bc45534a2d5`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:30.387453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:30.398030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1181/2373] 10.222.0.97 () {58 vars in 1058 bytes} [Thu Sep 8 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:07:30.410559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6eaa-0d39-4c73-b2dd-04985bad466a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:30.431585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97a6eaa-0d39-4c73-b2dd-04985bad466a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:30.439252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:30.439787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:30.440067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:30.440387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:30.450947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:30.453746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:30.466486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1193/2374] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:07:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:07:35.675724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:35.678598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:35.684843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=53 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:35.756974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead60d27-e4a6-4853-b2e6-ae74f04bf1a8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:35.762691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:35.763542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:35.763871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:35.764317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] PATH_INFO: `/v3/users/775ee674f5084d01b48d7b3ef20c7864` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:35.765705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=775ee674f5084d01b48d7b3ef20c7864)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:35.768198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:36.002108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775ee674f5084d01b48d7b3ef20c7864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.002864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775ee674f5084d01b48d7b3ef20c7864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.011630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775ee674f5084d01b48d7b3ef20c7864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.026944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775ee674f5084d01b48d7b3ef20c7864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.027727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775ee674f5084d01b48d7b3ef20c7864'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.164081 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1182/2375] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:35 2022] DELETE /identity/v3/users/775ee674f5084d01b48d7b3ef20c7864 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:36.503507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:36.528533 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31dbd1bd-4c54-4ca9-b4ac-80d520eff90f service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:36.599580 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:36.626371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:36.627039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:36.627336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:36.627608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] PATH_INFO: `/v3/projects/9aa00aef04784c1ea61221c69e872045` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:36.628880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9aa00aef04784c1ea61221c69e872045)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:36.631168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:36.791262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9aa00aef04784c1ea61221c69e872045', 'request_id': 'req-1fa2d605-0214-4e47-972b-7fd2ad957d1f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.798742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9aa00aef04784c1ea61221c69e872045', 'request_id': 'req-1fa2d605-0214-4e47-972b-7fd2ad957d1f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.805617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9aa00aef04784c1ea61221c69e872045', 'request_id': 'req-1fa2d605-0214-4e47-972b-7fd2ad957d1f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.899344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] The token cache is being invalidate because project 9aa00aef04784c1ea61221c69e872045 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:36.899344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.899635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:36.904118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1194/2376] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:36 2022] DELETE /identity/v3/projects/9aa00aef04784c1ea61221c69e872045 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:36.911797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:36.916586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:36.942986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae55a5e7-e5ce-4ccb-afd6-51bfe4bde6ac admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:37.048869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:37.049514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:37.049859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:37.050065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] PATH_INFO: `/v3/users/5d8edf055ab9430ca2af83e70ad27f66` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:37.051298 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8edf055ab9430ca2af83e70ad27f66)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:37.053252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:37.079356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:37.275138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8edf055ab9430ca2af83e70ad27f66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.276031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8edf055ab9430ca2af83e70ad27f66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.286479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8edf055ab9430ca2af83e70ad27f66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.309479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8edf055ab9430ca2af83e70ad27f66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.310563 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-065410c8-8be5-4ebe-94f1-93a63f076e5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8edf055ab9430ca2af83e70ad27f66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:37.412805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1183/2377] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:07:36 2022] DELETE /identity/v3/users/5d8edf055ab9430ca2af83e70ad27f66 => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:37.744259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:07:37.752260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:37.789137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa2d605-0214-4e47-972b-7fd2ad957d1f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:07:37.918562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:07:37.919620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:37.920070 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:37.920571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] PATH_INFO: `/v3/projects/df63dfc21ddb4ed596a16254145a59e4` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:37.922505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df63dfc21ddb4ed596a16254145a59e4)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:07:37.925814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:07:38.326207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df63dfc21ddb4ed596a16254145a59e4', 'request_id': 'req-b02a6bf2-4188-4c85-8963-bc5a49340777'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.333379 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df63dfc21ddb4ed596a16254145a59e4', 'request_id': 'req-b02a6bf2-4188-4c85-8963-bc5a49340777'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.339623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df63dfc21ddb4ed596a16254145a59e4', 'request_id': 'req-b02a6bf2-4188-4c85-8963-bc5a49340777'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.561593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:38.566071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] The token cache is being invalidate because project df63dfc21ddb4ed596a16254145a59e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:07:38.569774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.569774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:07:38.569774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1195/2378] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:07:37 2022] DELETE /identity/v3/projects/df63dfc21ddb4ed596a16254145a59e4 => generated 0 bytes in 826 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:07:47.407945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=20,keystone:DELETE=6,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:07:49.381694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944724a-0221-44a0-9f8a-a3fb67115090 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:07:49.382976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944724a-0221-44a0-9f8a-a3fb67115090 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:07:49.382976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6944724a-0221-44a0-9f8a-a3fb67115090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:07:49.403840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6944724a-0221-44a0-9f8a-a3fb67115090 None None] MFA Rules not processed for user `9b0569d9de724d5fbe3f41cbc0687d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:07:49.405018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6944724a-0221-44a0-9f8a-a3fb67115090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:07:49.468086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1184/2379] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:07:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:07:59.461088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=25 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:00.433657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:00.436904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:00.617234 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a6bf2-4188-4c85-8963-bc5a49340777 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:00.622474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:00.623521 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:00.624035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:00.624762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:00.627210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:00.858215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:00.862278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0b3df2-10c6-4d7e-ac23-797036e5985a service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:00.920904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1196/2380] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:00.926694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4918a-1080-4718-9b5a-c2bb04d137a5 None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:00.928497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4918a-1080-4718-9b5a-c2bb04d137a5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:00.928711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f4918a-1080-4718-9b5a-c2bb04d137a5 None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:00.931348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1185/2381] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:08:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:08:00.938652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfa57-813f-4806-84e4-ee20c43718c6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:00.938943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfa57-813f-4806-84e4-ee20c43718c6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:00.939218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291dfa57-813f-4806-84e4-ee20c43718c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:00.975968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:01.047618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-291dfa57-813f-4806-84e4-ee20c43718c6 None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:01.049766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-291dfa57-813f-4806-84e4-ee20c43718c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:01.076969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1197/2382] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:08:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:01.132694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4918a-1080-4718-9b5a-c2bb04d137a5 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:01.146671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f4918a-1080-4718-9b5a-c2bb04d137a5 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:01.151684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:01.155339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:01.155339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:01.155339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:01.173916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:01.175609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995ffeb0-e087-4948-a5f4-c7676a8b6722 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:01.201469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1186/2383] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:01.525442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc0d52-f8a3-4f40-a2f0-f8306b2e23f4 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:01.526170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc0d52-f8a3-4f40-a2f0-f8306b2e23f4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:01.526170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc0d52-f8a3-4f40-a2f0-f8306b2e23f4 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:01.527410 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1198/2384] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:08:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:08:01.539348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651bc-b543-4100-a40c-4f9b685f7901 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:01.539973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651bc-b543-4100-a40c-4f9b685f7901 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:01.540619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c651bc-b543-4100-a40c-4f9b685f7901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:01.565698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-46c651bc-b543-4100-a40c-4f9b685f7901 None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:01.567413 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-46c651bc-b543-4100-a40c-4f9b685f7901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:01.705231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1187/2385] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:08:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:01.755227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc0d52-f8a3-4f40-a2f0-f8306b2e23f4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:01.772237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc0d52-f8a3-4f40-a2f0-f8306b2e23f4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:01.779591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:01.780725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:01.781210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:01.781698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:01.806980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:01.809783 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:01.829595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1199/2386] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:02.533444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:02.534956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:02.541039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:02.587511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] MFA Rules not processed for user `26aa79753d6e4e49ad66cc32ba786fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:02.589082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:02.684241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1188/2387] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:02 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:06.777928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:06.785524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:06.820517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9f62e7-fefe-4bb1-a0e2-d5c102726e88 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:06.831126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:06.832089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:06.832610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:06.833088 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] PATH_INFO: `/v3/users/a30b0734975d4e5a848be44aa7860807` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:06.835000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a30b0734975d4e5a848be44aa7860807)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:06.837018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.115307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a30b0734975d4e5a848be44aa7860807'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.115985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a30b0734975d4e5a848be44aa7860807'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.124053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a30b0734975d4e5a848be44aa7860807'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.125438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a30b0734975d4e5a848be44aa7860807'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.140437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-15140069-0f5e-498d-9270-4822fc4bf44c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a30b0734975d4e5a848be44aa7860807'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.243578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1200/2388] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:06 2022] DELETE /identity/v3/users/a30b0734975d4e5a848be44aa7860807 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:07.559224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:07.577197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fe14a1-8b5b-4c37-bfd6-432017676e48 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:07.680566 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:07.681334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:07.681719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:07.682094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] PATH_INFO: `/v3/projects/44e7a5fc099c4041aa1a3109962b5dc3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:07.683523 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e7a5fc099c4041aa1a3109962b5dc3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:07.685594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:07.868314 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:07.881113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e7a5fc099c4041aa1a3109962b5dc3', 'request_id': 'req-4088ba1b-dd72-40d2-825a-f707d6a24edc'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.888069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e7a5fc099c4041aa1a3109962b5dc3', 'request_id': 'req-4088ba1b-dd72-40d2-825a-f707d6a24edc'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:07.894203 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e7a5fc099c4041aa1a3109962b5dc3', 'request_id': 'req-4088ba1b-dd72-40d2-825a-f707d6a24edc'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.062508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] The token cache is being invalidate because project 44e7a5fc099c4041aa1a3109962b5dc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:08.062860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.063570 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4088ba1b-dd72-40d2-825a-f707d6a24edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:08.065796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1189/2389] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:07 2022] DELETE /identity/v3/projects/44e7a5fc099c4041aa1a3109962b5dc3 => generated 0 bytes in 507 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:08.072027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.073771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.073771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.151728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:08.153041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.220421 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1201/2390] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:08.228815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.229125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.229452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.254376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:08.255700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:08.264293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1190/2391] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:08:08.272268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.283686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8080da-7ca4-4dc3-a618-68d4e43d33de None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.289221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.289823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.290102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.290377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.291140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:08:08.291454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.294077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.309318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1202/2392] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:08:08.314668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.327979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee080f3-6f97-43a9-9665-850a163a3484 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.332952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.333556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.333845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.334602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.335245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.337539 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.509872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1191/2393] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/projects => generated 387 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:08.514116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.525001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aaaf4c-e16c-4bf5-8eb9-7b540bb33421 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.537866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:08.591312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.591915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.592254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.592542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.593165 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.595206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.728884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1203/2394] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/users => generated 294 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:08:08.733219 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.743698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb730ac-4243-42e6-94fb-ebaeab56b275 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.748443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.749040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.749344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.749621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.750269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.751874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.764302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1192/2395] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:08.767039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.778377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f1a698-915d-4baf-9454-aa8b8cbb7752 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.783148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.783832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.784136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.784498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.785134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.786824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eef075-ac97-4120-a53a-b4eba45d975d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.799197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1204/2396] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:08:08.803670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:08.814288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd1dd3c-870b-4725-aae4-183fc72bf544 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:08.820028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:08.820725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.821133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.821286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] PATH_INFO: `/v3/projects/1abe0ab206094524af0fdde9e6285385/users/f5e920d3b3044859b767add41690043c/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:08.837807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1abe0ab206094524af0fdde9e6285385, user_id=f5e920d3b3044859b767add41690043c, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:08.840963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:08.974904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1193/2397] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:08:08 2022] PUT /identity/v3/projects/1abe0ab206094524af0fdde9e6285385/users/f5e920d3b3044859b767add41690043c/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:08.980102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:08.980404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:08.980682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:09.001545 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:09.002754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:09.105946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1205/2398] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:09.120051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:09.122743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:09.204924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70d721b-abf1-490f-94a0-f78a33be326e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:09.208944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:09.210086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:09.210358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:09.210635 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:09.221716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:09.227969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d2e433-dd14-44ed-ae65-f5242d291b5c service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:09.236433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1194/2399] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:14.982990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:14.985723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.082350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e76b0e-052b-4504-bc4a-94049a42f8ff None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:15.093001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:15.093001 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:15.093879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:15.093879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:15.095803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:15.110230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:15.111883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:15.125066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1206/2400] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:18.482871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6a217-87fb-4e6f-8406-c660d43f4c3f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.483422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6a217-87fb-4e6f-8406-c660d43f4c3f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.483621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f6a217-87fb-4e6f-8406-c660d43f4c3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.514960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f4f6a217-87fb-4e6f-8406-c660d43f4c3f None None] MFA Rules not processed for user `f5e920d3b3044859b767add41690043c`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:18.516249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f4f6a217-87fb-4e6f-8406-c660d43f4c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:18.551360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:18.586148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1195/2401] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:18 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:18.601102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:18.613136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0101ccd6-c467-4d61-9692-36de1dd4e3e0 service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:18.619573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:18.620215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:18.620501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:18.620793 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:18.629582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:18.631864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:18.643856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1207/2402] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:22.136786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:22.137316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:22.137641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:22.161827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] MFA Rules not processed for user `f5e920d3b3044859b767add41690043c`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:22.162383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:22.170697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1196/2403] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:22 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:22.553443 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:22.555046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:22.563496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:22.567624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485a671-15f1-48a5-ad42-45bd4686d14a service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:22.572713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:22.573351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:22.573634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:22.573968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:22.575885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:22.586403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:22.587861 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:22.596400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1208/2404] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:32.153879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=15 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:32.898751 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:08:54.851713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:54.861612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f621b530-1925-4682-9754-f8bb40ec774a None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:54.865274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:54.865862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:54.866151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:54.866407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:54.868105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:54.880339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:54.881748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:54.891444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1197/2405] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:08:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:08:58.933291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:58.938609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:58.958397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c052e8e5-76a7-4aba-950e-acdadc585fc2 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:58.963946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:58.964616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:58.964911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:58.965185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] PATH_INFO: `/v3/users/c18dc6b5898c4e1aae71a028b4840bf0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:58.966428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c18dc6b5898c4e1aae71a028b4840bf0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:58.968415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:59.227660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c18dc6b5898c4e1aae71a028b4840bf0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.234176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c18dc6b5898c4e1aae71a028b4840bf0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.236755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c18dc6b5898c4e1aae71a028b4840bf0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.237445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c18dc6b5898c4e1aae71a028b4840bf0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.251328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-93db0dcc-35e0-44ec-aec3-5e494193e9bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c18dc6b5898c4e1aae71a028b4840bf0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.331020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1209/2406] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:08:58 2022] DELETE /identity/v3/users/c18dc6b5898c4e1aae71a028b4840bf0 => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:59.590205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:08:59.601153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b86426-7f3a-4022-889d-02b33a00c8a5 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:08:59.662205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:08:59.662898 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:59.663187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:59.663463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] PATH_INFO: `/v3/projects/73ba0ac1c50e4802b8c055a9b3075022` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:59.664645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73ba0ac1c50e4802b8c055a9b3075022)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:08:59.666547 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:08:59.767851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73ba0ac1c50e4802b8c055a9b3075022', 'request_id': 'req-d3af772d-d404-4156-b5c0-3747083d7945'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.775800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73ba0ac1c50e4802b8c055a9b3075022', 'request_id': 'req-d3af772d-d404-4156-b5c0-3747083d7945'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.781634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73ba0ac1c50e4802b8c055a9b3075022', 'request_id': 'req-d3af772d-d404-4156-b5c0-3747083d7945'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.857187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] The token cache is being invalidate because project 73ba0ac1c50e4802b8c055a9b3075022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:08:59.857554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.858288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d3af772d-d404-4156-b5c0-3747083d7945 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:08:59.859881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1198/2407] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:08:59 2022] DELETE /identity/v3/projects/73ba0ac1c50e4802b8c055a9b3075022 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:08:59.866007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:59.866387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:59.866709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:08:59.889141 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:08:59.890390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:08:59.970882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1210/2408] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:08:59.979895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:08:59.980344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:08:59.980449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.002179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:00.003493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.012941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1199/2409] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:00.017701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.031021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314f5461-3ae8-471b-929e-e63c2f73119d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.037662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.037662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.037662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.037662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.038218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:00.038526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.040195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.053534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1211/2410] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:00.058743 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.068646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88b54d0-52bd-4992-aafc-bcede0746ac6 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.074347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.075067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.075465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.083487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.083487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.083487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.219894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1200/2411] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:00 2022] POST /identity/v3/projects => generated 385 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:00.225520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.236940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859cd1d-8563-4a9a-adb6-1460b4368051 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.300634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.301151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.301432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.301739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.302359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.304427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.312666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:00.435208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1212/2412] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:00 2022] POST /identity/v3/users => generated 293 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:00.439375 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.451542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf277a3-0c5d-4e24-b143-0e4b6b5a6e80 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.458527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.459143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.460181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.460181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.460455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.462030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.473106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1201/2413] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:00.477130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.487444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c346748f-44e0-4bdb-8853-039ce39369f8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.492494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.492742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.493022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.493293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.493937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.495518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e315d0-9b41-4400-b473-6e08bbfb4ab9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.506585 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1213/2414] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:00.510960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.521666 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3acb32-fec9-47f3-badb-e6ddcf2d6849 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.526687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.527335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.527616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.527890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] PATH_INFO: `/v3/projects/053e4fab2bf84372bdaa118a9d6d8638/users/33fc47c3581b4211a83b9fff47789946/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.543247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=053e4fab2bf84372bdaa118a9d6d8638, user_id=33fc47c3581b4211a83b9fff47789946, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.545770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.618262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1202/2415] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:00 2022] PUT /identity/v3/projects/053e4fab2bf84372bdaa118a9d6d8638/users/33fc47c3581b4211a83b9fff47789946/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:00.631480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.631848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.632112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.665596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] MFA Rules not processed for user `33fc47c3581b4211a83b9fff47789946`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:00.666872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.735437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1214/2416] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:00 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:00.748768 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:00.750522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:00.766106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:00.828678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a9c53-be4e-4efd-9907-0fd9efdfcde7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:00.834430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:00.835035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:00.835323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:00.835603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:00.843473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:00.845112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b03fc6fa-3622-4439-8210-4b107bcc8932 service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:00.854103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1203/2417] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:10.553237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:10.647466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57d37b3-a69e-47c7-8b9f-03ca07417bdf None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:10.652556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:10.653281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:10.653618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:10.653954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] PATH_INFO: `/v3/users/33fc47c3581b4211a83b9fff47789946` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:10.655256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33fc47c3581b4211a83b9fff47789946)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:10.657403 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:10.936879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33fc47c3581b4211a83b9fff47789946'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:10.937489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33fc47c3581b4211a83b9fff47789946'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:10.949424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33fc47c3581b4211a83b9fff47789946'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:10.950596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33fc47c3581b4211a83b9fff47789946'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:10.972840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:10.983120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33fc47c3581b4211a83b9fff47789946'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:11.006670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=28 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:11.108155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1215/2418] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:10 2022] DELETE /identity/v3/users/33fc47c3581b4211a83b9fff47789946 => generated 0 bytes in 557 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:11.114416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.114910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.115403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.148202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:11.149421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.218633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1204/2419] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:11.229394 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:11.230798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.307654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225da14e-1f80-4c33-bb21-0cad665f8720 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:11.311646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:11.312298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.312575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.312846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.329900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:11.331452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24de104b-86c6-49a2-9578-7954a0915751 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:11.343733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1216/2420] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:11.855576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:11.857116 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:11.870714 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a4d001-266e-43ba-8668-fbacc3121025 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:11.877161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:11.877161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:11.877161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:11.877161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] PATH_INFO: `/v3/projects/053e4fab2bf84372bdaa118a9d6d8638` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:11.886614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] RBAC: Authorizing `identity:delete_project(project_id=053e4fab2bf84372bdaa118a9d6d8638)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:11.889166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.016480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '053e4fab2bf84372bdaa118a9d6d8638', 'request_id': 'req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.024306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '053e4fab2bf84372bdaa118a9d6d8638', 'request_id': 'req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.030207 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '053e4fab2bf84372bdaa118a9d6d8638', 'request_id': 'req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.158659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] The token cache is being invalidate because project 053e4fab2bf84372bdaa118a9d6d8638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:12.159018 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.160408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8d4bc4f4-f0b8-48f9-95dd-657b9bf4025e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:12.161953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1205/2421] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:11 2022] DELETE /identity/v3/projects/053e4fab2bf84372bdaa118a9d6d8638 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:12.167217 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.167764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.167764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.188556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:12.189849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:12.256746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1217/2422] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:12.264965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.265273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.265551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.286697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:12.287854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:12.296028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1206/2423] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:09:12.305827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.316551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04657bee-eb7f-4d6a-90a9-aa2ac37b19c6 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.321550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.322180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.322459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.322732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.323382 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:12.323685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.325347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.339399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1218/2424] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:12.344936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.354496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c379383-9b6c-4776-b8fe-87bad68e02cc None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.359388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.360012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.361878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.361878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.361878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.363155 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.482283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1207/2425] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/projects => generated 373 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:12.486738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.497255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28037602-1b94-4bfd-b179-895a195ac525 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.555611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.556113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.556426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.556703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.557344 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.559282 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.652801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1219/2426] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/users => generated 287 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:12.657106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.669065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c35d59-2c49-41e5-aa38-6ba44ca83057 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.674039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.674688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.675465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.675738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.676436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.678024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.688626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1208/2427] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:12.697177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.707680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac4df52-d678-4e11-acc1-6ca0bdeef61e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.712516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.713121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.713402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.713702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.714350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.716062 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcef9c37-0b23-4f36-aa45-2004e581aa23 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.717713 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:12.728060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1220/2428] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:12.735132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:12.746458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca40802-c949-43a9-8966-8c0d29b37258 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:12.751463 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:12.752291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.752579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.752928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] PATH_INFO: `/v3/projects/bd00d529e0744019b2367d9204e8d8ff/users/4ba5793a0e08492dacd9369c14a10788/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.770951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd00d529e0744019b2367d9204e8d8ff, user_id=4ba5793a0e08492dacd9369c14a10788, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:12.773662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:12.855971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1209/2429] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:12 2022] PUT /identity/v3/projects/bd00d529e0744019b2367d9204e8d8ff/users/4ba5793a0e08492dacd9369c14a10788/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:12.868646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:12.869029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:12.869390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:12.899123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] MFA Rules not processed for user `4ba5793a0e08492dacd9369c14a10788`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:12.901126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:12.978627 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1221/2430] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:12.983389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.007232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:13.055479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691754b-3b02-4f81-8fc0-cc41c0b4a556 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.060782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.061505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.061838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.062177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.062871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.064913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.214080 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1210/2431] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:12 2022] POST /identity/v3/projects => generated 373 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:13.225089 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.230166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52648bc7-52a2-406a-876c-44c8c0e1d8a4 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.306700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.307301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.307708 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.307985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.308647 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.310549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.738384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1222/2432] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:13 2022] POST /identity/v3/users => generated 293 bytes in 521 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:13.742646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.754674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe53d2-f993-422f-9160-0200dbb82a7b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.760216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.760896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.761197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.761466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.762113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.763753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:13.777367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1211/2433] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:13.781983 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:13.792727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df254ec-d665-45d0-9132-b8b9414eec98 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:13.797503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:13.798172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:13.798459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:13.798727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] PATH_INFO: `/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8/users/f7bc7e31c90a4ba9bd5988553e6f9f4e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:13.815146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349d51ca944b4b2283b4bb4bdba2e9a8, user_id=f7bc7e31c90a4ba9bd5988553e6f9f4e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:13.818452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.138250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1223/2434] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:13 2022] PUT /identity/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8/users/f7bc7e31c90a4ba9bd5988553e6f9f4e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:14.142358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.153361 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefb7bbe-90bd-49d5-9e22-22e26bebb6b4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.215121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.215738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.216015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.216337 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.216983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.218567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892dd392-f004-4408-a343-36ce9614964f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.229188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1212/2435] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:14.233917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.245307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3567d-ff52-459b-9b61-5b30aa058e08 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.250087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.250764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.251045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.251408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] PATH_INFO: `/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8/users/f7bc7e31c90a4ba9bd5988553e6f9f4e/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.254054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=349d51ca944b4b2283b4bb4bdba2e9a8, user_id=f7bc7e31c90a4ba9bd5988553e6f9f4e, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.256946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.268066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1224/2436] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:14 2022] PUT /identity/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8/users/f7bc7e31c90a4ba9bd5988553e6f9f4e/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:14.280013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.280319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.281316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.313133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] MFA Rules not processed for user `f7bc7e31c90a4ba9bd5988553e6f9f4e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:14.314376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.401973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1213/2437] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:14.414329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.416136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:14.469679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:14.510990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d416540b-29f0-4727-85de-96a4aaa7a0b5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.515227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.515930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.516247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.516525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.528445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.531506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.544726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1225/2438] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:14.770057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:14.779808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5671ce96-b703-4654-91ad-e1ace39dd8bc None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:14.783379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:14.784014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:14.784819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:14.785091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:14.797799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:14.846194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:14.848247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:14.857524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1214/2439] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:14 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:24.607143 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=18 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:24.863088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=21 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:26.015169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:26.016658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:26.079074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419d168a-7350-4786-9b89-46efa6be1cae service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:26.083416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:26.084044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:26.084358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:26.084633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:26.086373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:26.154196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:26.155997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:26.168703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1226/2440] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:09:30.657500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:30.660123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:30.677831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00443f61-a5a3-457c-bf20-003236ecf479 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:30.683366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:30.691037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:30.691037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:30.691037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] PATH_INFO: `/v3/users/9b0569d9de724d5fbe3f41cbc0687d75` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:30.691037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0569d9de724d5fbe3f41cbc0687d75)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:30.691037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:30.932554 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0569d9de724d5fbe3f41cbc0687d75'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:30.933010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0569d9de724d5fbe3f41cbc0687d75'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:30.949140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0569d9de724d5fbe3f41cbc0687d75'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:30.965216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0569d9de724d5fbe3f41cbc0687d75'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:30.965996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e35c357b-e75d-4b57-8cf2-40c82ce7d123 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0569d9de724d5fbe3f41cbc0687d75'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.109180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1215/2441] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:30 2022] DELETE /identity/v3/users/9b0569d9de724d5fbe3f41cbc0687d75 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:31.374094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:31.385728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5c2054-f14e-4621-a6bb-0a493e94582b service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:31.457093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:31.457738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:31.458020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:31.458294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] PATH_INFO: `/v3/projects/92a3e908b87b445b9674d48f9a61bf94` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:31.459584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92a3e908b87b445b9674d48f9a61bf94)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:31.465251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:31.644821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92a3e908b87b445b9674d48f9a61bf94', 'request_id': 'req-a4fc3c1d-809e-4778-89dd-ae9c88714da8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.647169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:31.651975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92a3e908b87b445b9674d48f9a61bf94', 'request_id': 'req-a4fc3c1d-809e-4778-89dd-ae9c88714da8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.658710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92a3e908b87b445b9674d48f9a61bf94', 'request_id': 'req-a4fc3c1d-809e-4778-89dd-ae9c88714da8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.790915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] The token cache is being invalidate because project 92a3e908b87b445b9674d48f9a61bf94 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:09:31.791309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.792008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a4fc3c1d-809e-4778-89dd-ae9c88714da8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:31.793657 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1227/2442] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:09:31 2022] DELETE /identity/v3/projects/92a3e908b87b445b9674d48f9a61bf94 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:31.798656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:31.800004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:31.800785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:31.832289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:31.834263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:31.910745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1216/2443] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:31.918780 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:31.919102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:31.919374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:31.944955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:31.947091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:31.960381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1228/2444] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:31.965282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:31.977623 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20cc1cd-363b-41d0-9d51-d200539c06f9 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:31.982625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:31.983282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:31.983562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:31.983835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:31.984624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:09:31.984856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:31.986569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:31.999559 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1217/2445] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:09:32.004355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.014293 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c632474-9e5e-47bf-97c6-685f1259332f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.019438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.020042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.020353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.020624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.021309 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.023277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.124947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1229/2446] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:32 2022] POST /identity/v3/projects => generated 357 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:32.129243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.140013 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805daa5-715a-4895-af42-cbf6b5fa8a4c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.218198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.219253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.219750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.220048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.221723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.225159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.239157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:32.346871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1218/2447] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:32 2022] POST /identity/v3/users => generated 279 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:09:32.351134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.362693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56225d8-f47d-4c57-a5fc-7ea0a03202af admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.368445 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.369086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.369374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.369648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.370299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.371918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.383701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1230/2448] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:32.387776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.398222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71fdc6-9e41-4f99-861c-69f8be627f6d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.402982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.403615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.404030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.404244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.405002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.406506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6597fa9-a6d7-424d-9b09-7c41b4c1416c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.418555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1219/2449] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:32.422732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.434324 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d69540-07e6-4aa1-9d0a-3c96117334fc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.439016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.439920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.440677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.440951 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] PATH_INFO: `/v3/projects/69d80feca2cf4e128c7f0701a09ab621/users/ca6e14d654914df18a18affc07ce0518/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.457216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=69d80feca2cf4e128c7f0701a09ab621, user_id=ca6e14d654914df18a18affc07ce0518, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.459882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.542174 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1231/2450] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:32 2022] PUT /identity/v3/projects/69d80feca2cf4e128c7f0701a09ab621/users/ca6e14d654914df18a18affc07ce0518/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:32.553328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.553610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.553891 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.583332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] MFA Rules not processed for user `ca6e14d654914df18a18affc07ce0518`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:32.584644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:32.655399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1220/2451] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:32.660612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.736023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54875702-f89e-4559-8dc1-1d7b862c9223 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.741681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.742438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.742727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.743005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.743644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.745593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:32.848256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1232/2452] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:09:32 2022] POST /identity/v3/projects => generated 357 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:32.852589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:32.863826 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764860eb-c830-4e80-b171-f8783dd85580 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:32.924736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:32.925349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:32.925630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:32.925903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:32.926546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:32.928501 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.022055 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1221/2453] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:09:32 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:09:33.027278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.038561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd1627-9f55-4323-ba2b-4376e491b3af admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.043317 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.044016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.044329 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.044603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.045263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.046857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.058368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1233/2454] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:33.062772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.074582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd26843-0c67-4e7d-8434-564e0e5be0fc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.079473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.080125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.080940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.081210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] PATH_INFO: `/v3/projects/edcc1abf46884b9da918aaf4a1486ff3/users/5c9a5c73501d45adb76d17128fc7b186/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.098098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcc1abf46884b9da918aaf4a1486ff3, user_id=5c9a5c73501d45adb76d17128fc7b186, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.100830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.212686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1222/2455] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:33 2022] PUT /identity/v3/projects/edcc1abf46884b9da918aaf4a1486ff3/users/5c9a5c73501d45adb76d17128fc7b186/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:33.216761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.227856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec8bf0-266c-462d-a153-d098c69a5117 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.277609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:33.288031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.288663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.288943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.289419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.289975 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.291574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf06c00-7102-4632-9baf-bf3d83b05ebd admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.301864 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1234/2456] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:09:33.307986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.319598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1434db1-698b-45a9-8c23-8a2c3fa97fd5 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.324526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.325297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.325574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.325844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] PATH_INFO: `/v3/projects/edcc1abf46884b9da918aaf4a1486ff3/users/5c9a5c73501d45adb76d17128fc7b186/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.327832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edcc1abf46884b9da918aaf4a1486ff3, user_id=5c9a5c73501d45adb76d17128fc7b186, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.331296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.342580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1223/2457] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:09:33 2022] PUT /identity/v3/projects/edcc1abf46884b9da918aaf4a1486ff3/users/5c9a5c73501d45adb76d17128fc7b186/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:09:33.354442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.354733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.355188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.387102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] MFA Rules not processed for user `5c9a5c73501d45adb76d17128fc7b186`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:33.388377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:33.465986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1235/2458] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:33.477647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.479383 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:33.504783 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:33.543077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe817bf5-5e81-438b-8659-75fb96bb37e6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.547061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.547683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.547966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.548269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.558557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.560040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.574118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1224/2459] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:33 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:33.825419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:33.836590 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bcc31e-f2a8-4f26-8033-4e5ae3e3ca52 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:33.841296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:33.842513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:33.842797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:33.843075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:33.906887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:33.908470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccb6f6f-2727-4273-8f71-29c69709511b service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:33.918124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1236/2460] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:33 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:43.734840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=17 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:43.914374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=68 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:45.987199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:45.988763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.048573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be13933-8107-4716-85ad-9e4af1894065 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:46.052053 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:46.052695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:46.052980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:46.053255 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:46.055071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:46.118029 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:46.119461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:46.130511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1225/2461] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:51.739538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:51.740034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:51.740391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:51.765661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:09:51.767035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:51.877256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1237/2462] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:09:52.128401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:52.129816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:52.203458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d418f23-21f2-4935-876e-a8eda0d27222 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:52.211802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:52.212656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:52.212936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:52.213211 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:52.224795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:52.227016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:52.243608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1226/2463] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:09:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:09:54.050564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:54.057499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:54.182782 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae61f29d-9106-46a3-a6f9-98532997789e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:54.189053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:54.189668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:54.189952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:54.190228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:54.195811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:54.209739 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:54.211343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:54.224391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1238/2464] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:55.486315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:55.498467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62d3af9-9e6c-49b5-9532-df872f49dcbd service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:55.504315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:55.505216 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:55.505506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:55.505769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:55.508494 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:09:55.565678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:55.590372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:55.591917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af4a3c-7c45-4cf3-ae2a-45bf15b8a11a service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:55.604354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1227/2465] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:09:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:09:59.385852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:09:59.423104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3246a167-57dc-4c96-86a3-4ae8ce525c36 service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:09:59.439650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:09:59.441246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:09:59.441993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:09:59.442656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] PATH_INFO: `/v3/users/ca6e14d654914df18a18affc07ce0518` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:09:59.447356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca6e14d654914df18a18affc07ce0518)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:09:59.454979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:09:59.886002 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca6e14d654914df18a18affc07ce0518'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:59.886858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca6e14d654914df18a18affc07ce0518'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:59.896368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca6e14d654914df18a18affc07ce0518'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:59.897258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca6e14d654914df18a18affc07ce0518'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:09:59.915307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:09:59.926476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca6e14d654914df18a18affc07ce0518'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:00.403363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1239/2466] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:09:59 2022] DELETE /identity/v3/users/ca6e14d654914df18a18affc07ce0518 => generated 0 bytes in 1016 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:00.405285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:00.406105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:00.407245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:00.473357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:00.475527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:00.648126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1228/2467] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:00.662825 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:00.673588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:00.875128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f8712-3e2f-46ee-967d-87fb07ae4c0e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:00.885270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:00.885969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:00.886190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:00.886464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:00.915124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:00.917326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:00.954404 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1240/2468] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:01.885015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:01.886682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:01.931661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08249286-d14f-4747-a6e9-eab6a0984630 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:01.945683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:01.946998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:01.947706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:01.948315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] PATH_INFO: `/v3/projects/69d80feca2cf4e128c7f0701a09ab621` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:01.950344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=69d80feca2cf4e128c7f0701a09ab621)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:01.952732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:02.155741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '69d80feca2cf4e128c7f0701a09ab621', 'request_id': 'req-bba71193-6777-4e89-8da2-927616bca5a4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.166853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '69d80feca2cf4e128c7f0701a09ab621', 'request_id': 'req-bba71193-6777-4e89-8da2-927616bca5a4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.178185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '69d80feca2cf4e128c7f0701a09ab621', 'request_id': 'req-bba71193-6777-4e89-8da2-927616bca5a4'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.359271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] The token cache is being invalidate because project 69d80feca2cf4e128c7f0701a09ab621 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:02.359832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.361254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.365278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1229/2469] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:01 2022] DELETE /identity/v3/projects/69d80feca2cf4e128c7f0701a09ab621 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:02.369655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:02.382742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9527010d-4c5a-4d85-9266-3bdfd9575625 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:02.483947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:02.484808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:02.485255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:02.485724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] PATH_INFO: `/v3/users/5c9a5c73501d45adb76d17128fc7b186` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:02.486912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c9a5c73501d45adb76d17128fc7b186)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:02.489260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:02.852643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c9a5c73501d45adb76d17128fc7b186'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.854909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c9a5c73501d45adb76d17128fc7b186'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.871847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c9a5c73501d45adb76d17128fc7b186'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.872773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c9a5c73501d45adb76d17128fc7b186'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.894518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-92cb9c8f-c145-4e23-9a91-0acfc1aa4204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c9a5c73501d45adb76d17128fc7b186'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:02.991341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1241/2470] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:02 2022] DELETE /identity/v3/users/5c9a5c73501d45adb76d17128fc7b186 => generated 0 bytes in 619 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:04.091540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:04.093312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:04.113663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba71193-6777-4e89-8da2-927616bca5a4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:04.200492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:04.201753 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:04.202045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:04.202320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] PATH_INFO: `/v3/projects/edcc1abf46884b9da918aaf4a1486ff3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:04.203842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edcc1abf46884b9da918aaf4a1486ff3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:04.207035 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:04.211023 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:04.388185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edcc1abf46884b9da918aaf4a1486ff3', 'request_id': 'req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:04.402197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edcc1abf46884b9da918aaf4a1486ff3', 'request_id': 'req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:04.413813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edcc1abf46884b9da918aaf4a1486ff3', 'request_id': 'req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:04.541989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] The token cache is being invalidate because project edcc1abf46884b9da918aaf4a1486ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:04.542409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:04.543326 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34cc3b70-9436-4c2d-bbd9-1337d4a8e971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:04.545636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1230/2471] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:04 2022] DELETE /identity/v3/projects/edcc1abf46884b9da918aaf4a1486ff3 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:04.552064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:04.552719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:04.553061 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:04.576610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:04.589444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:04.590666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:04.679697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1242/2472] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:04.688274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:04.688707 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:04.689565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:04.721793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:04.727817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:04.738606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1231/2473] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:04.743092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:04.754350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca676b48-4678-4b1b-a7df-e787e6e9819a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:04.759953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:04.760803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:04.761277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:04.761562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:04.762342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:04.762620 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:04.764662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:04.777831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1243/2474] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:10:04.786087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:04.797955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fab285-ac72-4848-b470-e5ecf11761d6 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:04.804456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:04.805082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:04.805364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:04.805641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:04.806364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:04.809827 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.176322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1232/2475] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:04 2022] POST /identity/v3/projects => generated 363 bytes in 391 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:05.180349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:05.194041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67b3935-4761-40d0-9823-f7ade8599aca admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:05.299759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:05.300741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.301198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.301566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.304245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:05.306947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.343032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:05.345190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:05.529942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957b648-5f36-440e-bd57-68625162d1b2 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:05.542961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:05.543874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.544885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.545291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.547947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:05.605113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1244/2476] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:05 2022] POST /identity/v3/users => generated 282 bytes in 422 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:05.606508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:05.623084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f18af7-59be-4e4a-b865-a27e9d9f96c1 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:05.632571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:05.633250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.633538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.633815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.634474 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:05.636224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.657208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1245/2477] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:05.658486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:05.659951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:05.662462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.674923 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cfb981-08ac-432f-9bfa-6a150e2dbb58 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:05.687575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:05.687575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.687575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.687575 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.689313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:05.689755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918230e0-10a1-4242-b604-4843f697d1a9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.693372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1233/2478] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:05 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:05.707460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1246/2479] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:05.708580 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:05.725777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9ebe5c-5ecb-4a7e-8b05-01f42b561756 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:05.734271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:05.736787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.738248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.738693 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] PATH_INFO: `/v3/projects/34edf1bfa6244564927fd79fa78eac75/users/44078bf2bb8c404da1424815a14e9d75/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.769120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34edf1bfa6244564927fd79fa78eac75, user_id=44078bf2bb8c404da1424815a14e9d75, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:05.774328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:05.884205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1234/2480] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:05 2022] PUT /identity/v3/projects/34edf1bfa6244564927fd79fa78eac75/users/44078bf2bb8c404da1424815a14e9d75/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:05.886037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:05.886402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:05.886684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:05.934684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] MFA Rules not processed for user `44078bf2bb8c404da1424815a14e9d75`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:05.938149 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:06.053604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1247/2481] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:06.066166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:06.138834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1057266a-6383-4511-aa69-84bcb3e84a25 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:06.145372 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:06.145973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:06.146252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:06.146526 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:06.149977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:06.151953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:06.365993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1235/2482] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:06 2022] POST /identity/v3/projects => generated 363 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:06.370246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:06.388520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c223748c-89d5-4d20-a42a-1d23b40d79f0 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:06.488757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:06.489375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:06.490274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:06.490274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:06.490600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:06.492542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:06.625203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1248/2483] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:06 2022] POST /identity/v3/users => generated 288 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:06.635177 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:06.649344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78125e43-16a9-4e11-ac88-372b8e48832f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:06.655711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:06.656622 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:06.657221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:06.658283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:06.659119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:06.660984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:06.684772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1236/2484] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:06.688253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:06.701307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5803e42-f31b-457d-a981-1cd0a7de9240 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:06.707680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:06.708725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:06.709004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:06.709269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] PATH_INFO: `/v3/projects/1af1fcae57c64b808fcf1d7bd222e910/users/03c987e55af94d2db05bc6fdf218ac72/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:06.726520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af1fcae57c64b808fcf1d7bd222e910, user_id=03c987e55af94d2db05bc6fdf218ac72, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:06.731294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:06.763830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:06.904588 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1249/2485] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:06 2022] PUT /identity/v3/projects/1af1fcae57c64b808fcf1d7bd222e910/users/03c987e55af94d2db05bc6fdf218ac72/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:06.909100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:06.920423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a694687-cfb8-46d2-b406-2b99a947ea7b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:07.017181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:07.022455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.023185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.033005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.033005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:07.033005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c850d4-f9f2-4101-b7b5-504b00b9e833 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:07.043323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1237/2486] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:06 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:07.048490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:07.061397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfb12f9-1724-49b8-9745-1dc706f08f0e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:07.073653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:07.079646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.079646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.079646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] PATH_INFO: `/v3/projects/1af1fcae57c64b808fcf1d7bd222e910/users/03c987e55af94d2db05bc6fdf218ac72/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.079646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af1fcae57c64b808fcf1d7bd222e910, user_id=03c987e55af94d2db05bc6fdf218ac72, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:07.083242 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:07.100775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1250/2487] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:07 2022] PUT /identity/v3/projects/1af1fcae57c64b808fcf1d7bd222e910/users/03c987e55af94d2db05bc6fdf218ac72/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:07.112394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.116652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.116652 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.150896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] MFA Rules not processed for user `03c987e55af94d2db05bc6fdf218ac72`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:07.150896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:07.222286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1238/2488] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:07 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:07.237291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:07.239398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:07.308102 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:07.310309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:07.316232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:07.319941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21840c7-e543-443d-8f4e-ecab1221ba6d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:07.326922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:07.327947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.328305 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.328582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.341757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:07.343249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:07.374804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1251/2489] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:07 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:07.414766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec07a43d-6eda-4945-8a60-943def6312fa None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:07.420269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:07.421071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.421562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.421842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] PATH_INFO: `/v3/users/1729bb4d783f425085185c344dfe2ab8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.429260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1729bb4d783f425085185c344dfe2ab8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:07.429260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:07.670258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:07.679352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86740ef8-0230-4c82-8c7e-f05b7e1c68c0 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:07.683027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:07.683618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:07.683901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:07.684221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:07.747458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:07.748946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:07.756159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1729bb4d783f425085185c344dfe2ab8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:07.756850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1729bb4d783f425085185c344dfe2ab8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:07.758653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1252/2490] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:07 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:07.764900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1729bb4d783f425085185c344dfe2ab8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:07.781840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1729bb4d783f425085185c344dfe2ab8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:07.785997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1729bb4d783f425085185c344dfe2ab8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:07.904792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1239/2491] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:07 2022] DELETE /identity/v3/users/1729bb4d783f425085185c344dfe2ab8 => generated 0 bytes in 597 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:08.244888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:08.261901 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc43f90d-c58d-46a0-beec-9efabb5662f4 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:08.343947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:08.345166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:08.345449 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:08.345719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] PATH_INFO: `/v3/projects/fe528c21e8c94eb6af8d5f86757328b8` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:08.346856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe528c21e8c94eb6af8d5f86757328b8)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:08.348797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:08.536850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe528c21e8c94eb6af8d5f86757328b8', 'request_id': 'req-d6eac039-963a-4fe4-b192-1809dcdf4968'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:08.545021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe528c21e8c94eb6af8d5f86757328b8', 'request_id': 'req-d6eac039-963a-4fe4-b192-1809dcdf4968'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:08.551056 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe528c21e8c94eb6af8d5f86757328b8', 'request_id': 'req-d6eac039-963a-4fe4-b192-1809dcdf4968'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:08.668429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] The token cache is being invalidate because project fe528c21e8c94eb6af8d5f86757328b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:08.668721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:08.673696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:08.673696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1253/2492] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:08 2022] DELETE /identity/v3/projects/fe528c21e8c94eb6af8d5f86757328b8 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:08.675390 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:08.677166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:08.697163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cee8468-fd2c-4ef1-9462-152260b9343c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:08.775507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:08.776156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:08.776461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:08.782749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] PATH_INFO: `/v3/users/82346818adfb42f2a1dfbbfc695e9e3f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:08.782749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=82346818adfb42f2a1dfbbfc695e9e3f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:08.782749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:09.112260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82346818adfb42f2a1dfbbfc695e9e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.112721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82346818adfb42f2a1dfbbfc695e9e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.120254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82346818adfb42f2a1dfbbfc695e9e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.139143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82346818adfb42f2a1dfbbfc695e9e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.139886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-39867b88-c995-40af-8e31-76304f915c3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82346818adfb42f2a1dfbbfc695e9e3f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.239889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:09.288933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1240/2493] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:08 2022] DELETE /identity/v3/users/82346818adfb42f2a1dfbbfc695e9e3f => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:09.422217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:09.554420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:09.555968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:09.571434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6eac039-963a-4fe4-b192-1809dcdf4968 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:09.638280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:09.638920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:09.639377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:09.639824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] PATH_INFO: `/v3/projects/586baaa868fa4165acc453adf529d01e` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:09.641253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=586baaa868fa4165acc453adf529d01e)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:09.643413 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:09.859444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '586baaa868fa4165acc453adf529d01e', 'request_id': 'req-a28231dd-d720-4799-bef0-c14e3fca19b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.867295 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '586baaa868fa4165acc453adf529d01e', 'request_id': 'req-a28231dd-d720-4799-bef0-c14e3fca19b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:09.874240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '586baaa868fa4165acc453adf529d01e', 'request_id': 'req-a28231dd-d720-4799-bef0-c14e3fca19b9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:10.008768 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] The token cache is being invalidate because project 586baaa868fa4165acc453adf529d01e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:10.009059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:10.014248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a28231dd-d720-4799-bef0-c14e3fca19b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:10.014248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1254/2494] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:09 2022] DELETE /identity/v3/projects/586baaa868fa4165acc453adf529d01e => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:10.793179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:10.993299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:16.925481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:16.931030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:16.931030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:16.931030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1241/2495] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:10:16.934693 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8de4d-9b77-4a1d-9a13-0c35c347441b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:16.935227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8de4d-9b77-4a1d-9a13-0c35c347441b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:16.936331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f8de4d-9b77-4a1d-9a13-0c35c347441b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:16.981021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-c4f8de4d-9b77-4a1d-9a13-0c35c347441b None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:16.982250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c4f8de4d-9b77-4a1d-9a13-0c35c347441b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.065495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1255/2496] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:10:16 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:17.081342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:17.083110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.154527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3661e2c4-df37-4eae-a481-80db9a0fa1cd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:17.158322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:17.158979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.159284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.159555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.170380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:17.171878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d883ace-e1ef-4eb1-bebc-6415160df751 service glance] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:17.184401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1242/2497] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:17.504115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.507201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.507201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] PATH_INFO: `/` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.507201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1256/2498] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:10:17.511758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1bbd-25ef-4804-8f8b-e817647acd62 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.512123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1bbd-25ef-4804-8f8b-e817647acd62 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.512581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb1bbd-25ef-4804-8f8b-e817647acd62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.534647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1ccb1bbd-25ef-4804-8f8b-e817647acd62 None None] MFA Rules not processed for user `f87892da36634672bac2c4c12a52f15e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:17.535847 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1ccb1bbd-25ef-4804-8f8b-e817647acd62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.594445 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1243/2499] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:17.607885 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:17.609899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.627523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757100b-9fe5-4c57-8f4b-faa5bfd74969 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:17.631700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:17.632347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.632641 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.632917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.645231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:17.645231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14446e88-e9a4-4dd2-809e-e724dd6452f1 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:17.653399 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1257/2500] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:17.761257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdbe9-fbf5-4e82-ae4d-8e321b725e2d None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.761548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdbe9-fbf5-4e82-ae4d-8e321b725e2d None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.761978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfdbe9-fbf5-4e82-ae4d-8e321b725e2d None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.764170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1244/2501] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 08 14:10:17.772177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22d8f-4fea-4820-b161-d13ffe261d3b None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.772559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22d8f-4fea-4820-b161-d13ffe261d3b None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.772932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b22d8f-4fea-4820-b161-d13ffe261d3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.793778 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-37b22d8f-4fea-4820-b161-d13ffe261d3b None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:17.795000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-37b22d8f-4fea-4820-b161-d13ffe261d3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:17.805347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1258/2502] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:10:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:17.824433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfdbe9-fbf5-4e82-ae4d-8e321b725e2d None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:17.839123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfdbe9-fbf5-4e82-ae4d-8e321b725e2d None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:17.842865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:17.843425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:17.843696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:17.843973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:17.859577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:17.862896 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:17.882174 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1245/2503] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:20.532807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700388b-54ba-4a45-acde-92aae3cf22f2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:20.533440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700388b-54ba-4a45-acde-92aae3cf22f2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:20.533599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8700388b-54ba-4a45-acde-92aae3cf22f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:20.560576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-8700388b-54ba-4a45-acde-92aae3cf22f2 None None] MFA Rules not processed for user `26aa79753d6e4e49ad66cc32ba786fe8`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:20.569665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-8700388b-54ba-4a45-acde-92aae3cf22f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:20.628703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:20.638599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1259/2504] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:21.982340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:21.983775 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:22.025014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:22.076391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7dbedb-716e-4147-878a-7084d9758b78 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:22.084114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:22.084986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:22.085485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:22.085874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] PATH_INFO: `/v3/users/4ba5793a0e08492dacd9369c14a10788` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:22.087126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ba5793a0e08492dacd9369c14a10788)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:22.090530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:22.601789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ba5793a0e08492dacd9369c14a10788'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:22.601789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ba5793a0e08492dacd9369c14a10788'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:22.601789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ba5793a0e08492dacd9369c14a10788'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:22.601789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ba5793a0e08492dacd9369c14a10788'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:22.602327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ba5793a0e08492dacd9369c14a10788'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:23.047998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1246/2505] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:21 2022] DELETE /identity/v3/users/4ba5793a0e08492dacd9369c14a10788 => generated 0 bytes in 1053 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:23.052025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322101f-ee2a-40cb-b6ba-fbd8e3103dfe None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:23.052581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322101f-ee2a-40cb-b6ba-fbd8e3103dfe None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:23.052680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9322101f-ee2a-40cb-b6ba-fbd8e3103dfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:23.071972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9322101f-ee2a-40cb-b6ba-fbd8e3103dfe None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:23.073196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9322101f-ee2a-40cb-b6ba-fbd8e3103dfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:23.139798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1260/2506] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:23.150263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:23.151763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:23.221126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b70bdad-e2b8-464e-818e-6c0663510c98 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:23.226241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:23.227085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:23.227481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:23.227867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:23.240548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:23.242667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:23.255154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1247/2507] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:23.534431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:23.534710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:23.534986 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:23.556351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:23.557760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:23.627798 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1261/2508] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:10:23 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:23.649048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:23.649048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:23.718904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:23.745672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4f2b85-e39d-40b8-840a-e2c2a466c200 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:23.752270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:23.756719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:23.756719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:23.756719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:23.763118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:23.764848 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:23.774075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1248/2509] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:10:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:10:23.900503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:23.902044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:23.927139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9628de4-6e6b-443e-acf9-55b50e91066f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:23.932127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:23.932805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:23.938478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:23.938478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] PATH_INFO: `/v3/projects/bd00d529e0744019b2367d9204e8d8ff` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:23.938478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd00d529e0744019b2367d9204e8d8ff)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:23.938478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:24.079417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd00d529e0744019b2367d9204e8d8ff', 'request_id': 'req-36db2b55-f363-45f3-9384-9c25b2751a53'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.098026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd00d529e0744019b2367d9204e8d8ff', 'request_id': 'req-36db2b55-f363-45f3-9384-9c25b2751a53'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.111127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd00d529e0744019b2367d9204e8d8ff', 'request_id': 'req-36db2b55-f363-45f3-9384-9c25b2751a53'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.213654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] The token cache is being invalidate because project bd00d529e0744019b2367d9204e8d8ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:24.214154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.217319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.219697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1262/2510] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:23 2022] DELETE /identity/v3/projects/bd00d529e0744019b2367d9204e8d8ff => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:24.223928 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:24.237245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0a12b-28a5-4edd-883f-e029bbe365bc service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:24.337415 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:24.338477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:24.338983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:24.339477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] PATH_INFO: `/v3/users/f7bc7e31c90a4ba9bd5988553e6f9f4e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:24.341270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7bc7e31c90a4ba9bd5988553e6f9f4e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:24.343587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:24.593726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7bc7e31c90a4ba9bd5988553e6f9f4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.595648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7bc7e31c90a4ba9bd5988553e6f9f4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.607825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7bc7e31c90a4ba9bd5988553e6f9f4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.628506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7bc7e31c90a4ba9bd5988553e6f9f4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.629541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-bb4ee572-018e-481e-b1dd-a94c528c1e22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7bc7e31c90a4ba9bd5988553e6f9f4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:24.763717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:24.765243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:24.781365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36db2b55-f363-45f3-9384-9c25b2751a53 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:24.788536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:24.789639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:24.789995 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:24.790271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] PATH_INFO: `/v3/users/44078bf2bb8c404da1424815a14e9d75` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:24.796427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] RBAC: Authorizing `identity:delete_user(user_id=44078bf2bb8c404da1424815a14e9d75)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:24.799369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:24.800872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1249/2511] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:24 2022] DELETE /identity/v3/users/f7bc7e31c90a4ba9bd5988553e6f9f4e => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:24.831250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:25.086728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44078bf2bb8c404da1424815a14e9d75'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.088912 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44078bf2bb8c404da1424815a14e9d75'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.102078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44078bf2bb8c404da1424815a14e9d75'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.103750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44078bf2bb8c404da1424815a14e9d75'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.127091 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44078bf2bb8c404da1424815a14e9d75'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.240741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1263/2512] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:24 2022] DELETE /identity/v3/users/44078bf2bb8c404da1424815a14e9d75 => generated 0 bytes in 477 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:25.245534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:25.247143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:25.247665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:25.293588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:25.295157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.411786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:25.429576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1250/2513] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:25.438333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:25.440066 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.527532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:25.530304 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.561581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41048f-960d-4cfa-a15b-1b85c93a5ab3 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:25.570204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:25.570776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:25.571105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:25.571328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] PATH_INFO: `/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:25.572575 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=349d51ca944b4b2283b4bb4bdba2e9a8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:25.574881 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:25.580681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352907fb-72a7-4e5c-ae8d-34c016dd009b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:25.591677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:25.592686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:25.593267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:25.593815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:25.617491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:25.617982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5125251-868b-4e76-9abd-72a61247a95c service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:25.631727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1264/2514] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:25.757353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '349d51ca944b4b2283b4bb4bdba2e9a8', 'request_id': 'req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.767670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '349d51ca944b4b2283b4bb4bdba2e9a8', 'request_id': 'req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.776505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '349d51ca944b4b2283b4bb4bdba2e9a8', 'request_id': 'req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.878966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] The token cache is being invalidate because project 349d51ca944b4b2283b4bb4bdba2e9a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:25.879342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.879984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9e3ad424-f31e-42db-bd52-d45f1ea17b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:25.882019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1251/2515] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:25 2022] DELETE /identity/v3/projects/349d51ca944b4b2283b4bb4bdba2e9a8 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:25.886918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:25.887325 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:25.887587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:25.919582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:25.920930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:25.995875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1265/2516] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:26.004476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.004838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.005596 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.052132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:26.056724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:26.071005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1252/2517] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:26.075745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.092827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038bfb5c-e78b-41ff-98a4-b6892b14179e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.092827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.092827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.092827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.093875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.093875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:26.093875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.095043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.114318 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1266/2518] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:10:26.117468 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.128344 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66359ad-9059-4fe7-9ea7-450d5083fd49 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.133598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.133598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.133982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.134159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.134760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.136749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.243733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.247703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:26.278743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee32c2e4-2515-4a09-a69c-8314b17b7be1 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.285724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.286803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.287154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.287498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] PATH_INFO: `/v3/projects/34edf1bfa6244564927fd79fa78eac75` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.294488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=34edf1bfa6244564927fd79fa78eac75)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.296604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.306980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1253/2519] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:26 2022] POST /identity/v3/projects => generated 379 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:26.326711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.343265 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:26.411947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb46668-278b-4d8a-934c-c49bc45d615d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.418784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.419427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.419902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.420123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.420938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.426187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.536624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34edf1bfa6244564927fd79fa78eac75', 'request_id': 'req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:26.544041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34edf1bfa6244564927fd79fa78eac75', 'request_id': 'req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:26.550476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34edf1bfa6244564927fd79fa78eac75', 'request_id': 'req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:26.608242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1254/2520] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:26 2022] POST /identity/v3/users => generated 290 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:26.612444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.624684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b79e2-4320-4387-9213-a5357ff2b328 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.677656 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:26.699175 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.700318 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.706537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.706537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.706537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.706537 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.718716 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1255/2521] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:26.722531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.734241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cf13d4-f546-4e06-963a-48f2215b50fb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.739497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.740118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.740432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.740713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.742124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.743902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.754930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] The token cache is being invalidate because project 34edf1bfa6244564927fd79fa78eac75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:26.755229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:26.756877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:26.759835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1267/2522] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:26 2022] DELETE /identity/v3/projects/34edf1bfa6244564927fd79fa78eac75 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:26.764646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.772126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:26.773036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1256/2523] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:26.773981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:26.775889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:26.792113 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7380f42-b488-4e56-9256-113e2508c4f3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.795845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11c87dc-05c4-4ab8-97b5-7bb95a820c7a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:26.798098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.798782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.799253 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.799534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] PATH_INFO: `/v3/projects/5ca5bf5c5a264063a4f720507f48a06f/users/25a9bb79ec58472b82724c2d3c58a74f/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.802700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:26.803879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.806346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.806852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] PATH_INFO: `/v3/users/03c987e55af94d2db05bc6fdf218ac72` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.808333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=03c987e55af94d2db05bc6fdf218ac72)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.811199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.831976 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ca5bf5c5a264063a4f720507f48a06f, user_id=25a9bb79ec58472b82724c2d3c58a74f, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:26.834834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed250f9b-e9ee-4acb-9b5e-a7736214392c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:26.926192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1257/2524] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:26 2022] PUT /identity/v3/projects/5ca5bf5c5a264063a4f720507f48a06f/users/25a9bb79ec58472b82724c2d3c58a74f/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:26.938398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:26.938687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:26.938959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:26.975843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] MFA Rules not processed for user `25a9bb79ec58472b82724c2d3c58a74f`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:26.977303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:27.058744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03c987e55af94d2db05bc6fdf218ac72'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:27.060256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03c987e55af94d2db05bc6fdf218ac72'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:27.068241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1258/2525] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:27.069771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.075736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03c987e55af94d2db05bc6fdf218ac72'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:27.078135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03c987e55af94d2db05bc6fdf218ac72'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:27.101897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03c987e55af94d2db05bc6fdf218ac72'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:27.152605 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03510b1a-5294-468b-9221-3fd3ac8a1334 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.159221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.164329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.164329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.164329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.164329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.164329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.227608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1268/2526] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:26 2022] DELETE /identity/v3/users/03c987e55af94d2db05bc6fdf218ac72 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:27.316241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1259/2527] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:27 2022] POST /identity/v3/projects => generated 379 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:27.326926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.328549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:27.403058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cab270-1e18-42ff-9d56-4042d4ebeb6d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.447831 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:27.489378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.490007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.490288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.490563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.491213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.493209 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.605412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1269/2528] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:27 2022] POST /identity/v3/users => generated 296 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:27.610236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.621145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddc0fd5-fbfd-4878-a19e-eca00e8f5bb8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.629660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.630281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.630562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.630849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.631687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.633434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.649595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1260/2529] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:27.653652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.674628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5335be7-61b4-4021-a994-c45ad4a485bb admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.694648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.694648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.694648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.694648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] PATH_INFO: `/v3/projects/6c1962c3350c4a0b9e9abba842223b4e/users/81ac80f7256e4f7bb8b9ad24c6b61233/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.722074 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1962c3350c4a0b9e9abba842223b4e, user_id=81ac80f7256e4f7bb8b9ad24c6b61233, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.725067 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.782286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.784195 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:27.805880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add548bf-bd9a-483e-88ea-f2775d3837be admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.805880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.805880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.805880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.806321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] PATH_INFO: `/v3/projects/1af1fcae57c64b808fcf1d7bd222e910` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.806321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af1fcae57c64b808fcf1d7bd222e910)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.808112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.808943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1270/2530] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:27 2022] PUT /identity/v3/projects/6c1962c3350c4a0b9e9abba842223b4e/users/81ac80f7256e4f7bb8b9ad24c6b61233/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:27.813131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.813327 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:27.916018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452af712-b067-4f1f-8660-aacde9e34b34 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.921635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.922261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.922437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.922755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.923390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.924960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.936557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1271/2531] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:27 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:27.940381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:27.951666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad1f29-ce53-4d47-9ba1-8f884fcc5982 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:27.959486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:27.960439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.960818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.961192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] PATH_INFO: `/v3/projects/6c1962c3350c4a0b9e9abba842223b4e/users/81ac80f7256e4f7bb8b9ad24c6b61233/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.963364 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1962c3350c4a0b9e9abba842223b4e, user_id=81ac80f7256e4f7bb8b9ad24c6b61233, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:27.966760 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31024ec4-db5d-4b25-aacf-30188da02d58 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:27.983562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1272/2532] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:27 2022] PUT /identity/v3/projects/6c1962c3350c4a0b9e9abba842223b4e/users/81ac80f7256e4f7bb8b9ad24c6b61233/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:27.994615 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9023-d3e1-4a34-b075-6d2012e6a48e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:27.995182 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9023-d3e1-4a34-b075-6d2012e6a48e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:27.995608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536c9023-d3e1-4a34-b075-6d2012e6a48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:27.998154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af1fcae57c64b808fcf1d7bd222e910', 'request_id': 'req-e87de37e-f7d8-4478-be96-0c46e595caaa'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:28.014564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af1fcae57c64b808fcf1d7bd222e910', 'request_id': 'req-e87de37e-f7d8-4478-be96-0c46e595caaa'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:28.037260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af1fcae57c64b808fcf1d7bd222e910', 'request_id': 'req-e87de37e-f7d8-4478-be96-0c46e595caaa'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:28.065258 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-536c9023-d3e1-4a34-b075-6d2012e6a48e None None] MFA Rules not processed for user `81ac80f7256e4f7bb8b9ad24c6b61233`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:28.066889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-536c9023-d3e1-4a34-b075-6d2012e6a48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.174618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1273/2533] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:27 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:28.178278 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] The token cache is being invalidate because project 1af1fcae57c64b808fcf1d7bd222e910 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:28.178587 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:28.179266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:28.180981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1261/2534] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:27 2022] DELETE /identity/v3/projects/1af1fcae57c64b808fcf1d7bd222e910 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:28.185865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db63ea-5f88-47f6-a497-7a8ecee53b54 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.186150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db63ea-5f88-47f6-a497-7a8ecee53b54 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.186543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20db63ea-5f88-47f6-a497-7a8ecee53b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.193247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.195025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.196887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:28.226439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-20db63ea-5f88-47f6-a497-7a8ecee53b54 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:28.228213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-20db63ea-5f88-47f6-a497-7a8ecee53b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.281300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87de37e-f7d8-4478-be96-0c46e595caaa admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.306116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1274/2535] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:28.317340 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.317945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.318573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.344711 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.345490 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.352421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.352421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.352805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:28.352805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.354259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.363939 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1275/2536] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:28.367776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.373671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.375961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.379361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35af59fb-9614-43ed-94e6-3a539ef10307 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.387054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.387779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.388071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1262/2537] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:28 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:28.388270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.388452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.389026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:28.389331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.391027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.406247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1276/2538] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:10:28.410444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.428657 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff32bd75-d882-453f-ade1-1a65efb34eea service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.435507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.443667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.443667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.443667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.443667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.443667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.545028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1263/2539] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:28 2022] POST /identity/v3/projects => generated 387 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:28.549758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.560998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a171f56-3033-4b19-baa3-d71f48543384 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.633944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.634762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.635043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.635430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.636063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.638332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.664771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.703604 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:28.744444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b1ced1-32ed-43bd-848e-6efd1c7e95db admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.748440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.749142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.749498 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.749850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.752270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:28.770308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1277/2540] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:28 2022] POST /identity/v3/users => generated 294 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:28.775239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.805629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5b136f-adee-4e39-93d4-389f85193b57 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.833700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.834636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.835025 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.835405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.836422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.838872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.865192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1278/2541] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:28.866150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.867205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.867612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.880681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1264/2542] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:28 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:28.903848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f280c4cd-e8bf-4df6-9efa-4fb3a33cbfc4 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.917241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.918377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.918791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.920069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:28.920925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:28.923312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5daa72-bcb9-4326-b96c-945b631b8d14 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:28.939829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1279/2543] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:28.941819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:28.973317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387d140-59d1-4ec2-9754-7de662e918c7 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:28.980985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:28.987107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:28.987107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:28.987107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] PATH_INFO: `/v3/projects/357cd4132a814121ad5c62c6db670c10/users/9b0f802befa642a59860da680ff548a5/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.015422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=357cd4132a814121ad5c62c6db670c10, user_id=9b0f802befa642a59860da680ff548a5, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:29.018813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:29.150232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1265/2544] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:28 2022] PUT /identity/v3/projects/357cd4132a814121ad5c62c6db670c10/users/9b0f802befa642a59860da680ff548a5/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:29.162584 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.163219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.165339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.201330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] MFA Rules not processed for user `9b0f802befa642a59860da680ff548a5`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:29.202614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:29.277823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:29.311434 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1280/2545] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:29 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:29.315850 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:29.409766 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02d2909-f448-4164-8be4-474bdc01ac5e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:29.415569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:29.416224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.416514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.416795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.417444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:29.419840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:29.667181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1266/2546] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:29 2022] POST /identity/v3/projects => generated 387 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:29.680299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:29.692805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a35fc-22ed-48f3-b55c-19458242661e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:29.756224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:29.756843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.757136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.757629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.758304 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:29.760570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:29.876487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1281/2547] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:29 2022] POST /identity/v3/users => generated 294 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:29.880790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:29.896330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8a365b-36c6-4f26-a903-b8d345316382 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:29.902664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:29.903522 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.903832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.904228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.904962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:29.906844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:29.923723 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1267/2548] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:29.925419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:29.937418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b57164-69e5-48af-9ead-a1e8ac717894 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:29.943785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:29.944430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.944709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.944988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:29.945634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:29.947269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510e3819-3e13-43ea-a60f-e10982f86059 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:29.961950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1282/2549] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:29.963046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:29.975014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddb429-aeae-4a02-9af2-6421be7452fa admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:29.981011 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:29.981991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:29.982598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:29.983086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] PATH_INFO: `/v3/projects/67894b3b17e74bc8985ee5f61fe8b692/users/d5a96c5c41ae4c2fb66f8cf5bf9a3f03/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.001209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67894b3b17e74bc8985ee5f61fe8b692, user_id=d5a96c5c41ae4c2fb66f8cf5bf9a3f03, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:30.004132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:30.182128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1268/2550] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:29 2022] PUT /identity/v3/projects/67894b3b17e74bc8985ee5f61fe8b692/users/d5a96c5c41ae4c2fb66f8cf5bf9a3f03/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:30.192803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:30.193140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:30.193600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.226764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] MFA Rules not processed for user `d5a96c5c41ae4c2fb66f8cf5bf9a3f03`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:30.228019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:30.311948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1283/2551] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:30 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:30.318087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:30.328981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:30.403176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a89edca-0085-4cda-895e-1f0545eb0da9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:30.408583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:30.409214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:30.409495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:30.409816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.410550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:30.412412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:30.544206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1269/2552] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:30 2022] POST /identity/v3/projects => generated 381 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:30.546665 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:30.559263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a92a035-658e-4a4a-8cef-c63dbf7f5ed2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:30.616568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:30.628962 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:30.629660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:30.629942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:30.630212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.630933 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:30.634776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:30.748361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1284/2553] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:30 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:30.749708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:30.761974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aae4bd-88c0-404f-8066-022ea433ffab admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:30.767838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:30.768749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:30.769188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:30.769978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.771434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:30.773269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:30.806247 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1270/2554] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:30 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:30.807126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:30.824405 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb06f4-aff1-49fa-ac98-b75ce05ec22d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:30.832469 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:30.833501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:30.833829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:30.834249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] PATH_INFO: `/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6/users/a8f8f809dc1d4299a234cee2b03eb002/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:30.854257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8b95133bcf48e387993bf8fd66c4c6, user_id=a8f8f809dc1d4299a234cee2b03eb002, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:30.857228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:31.412353 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1285/2555] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:30 2022] PUT /identity/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6/users/a8f8f809dc1d4299a234cee2b03eb002/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:31.417058 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:31.435568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90fd361-a34c-4897-81e9-05335aaf2cfa admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:31.518454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:31.519329 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:31.519630 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:31.520077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:31.520837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:31.522719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb5073-3f1e-4ecd-a7a9-210c6fb37592 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:31.535990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1271/2556] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:31 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:31.540504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:31.553102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1021c95e-9eb2-4cfc-a7a7-5030d32758f8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:31.558276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:31.559013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:31.559357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:31.559699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] PATH_INFO: `/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6/users/a8f8f809dc1d4299a234cee2b03eb002/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:31.561721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6b8b95133bcf48e387993bf8fd66c4c6, user_id=a8f8f809dc1d4299a234cee2b03eb002, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:31.564680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:31.578926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1286/2557] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:31 2022] PUT /identity/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6/users/a8f8f809dc1d4299a234cee2b03eb002/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:31.590264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:31.590555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:31.590837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:31.640349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] MFA Rules not processed for user `a8f8f809dc1d4299a234cee2b03eb002`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:31.642594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:31.749117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1272/2558] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:31 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:31.757515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:31.841186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b8dc6a-a9e6-4a1b-93dd-68ce1bfd626f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:31.846432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:31.847322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:31.847900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:31.848351 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:31.871619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:31.873244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:31.886875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1287/2559] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:31 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:33.638670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:33.644349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:33.649117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5765ac7b-3532-4502-a309-0c02c5a9d580 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:33.652749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:33.653367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:33.653644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:33.653913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:33.714943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:33.716420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb328bee-2cb1-4da5-b4fe-d2750c38738c service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:33.727994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1273/2560] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:33 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:41.883004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=47,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:42.238028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:42.246604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e41bcf-d639-4478-9602-5c3bb50f131f service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:42.250312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:42.250947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:42.251224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:42.251494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:42.311201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:42.312710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:42.322571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1288/2561] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:45.092767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:45.093646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:45.093816 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:45.119210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:45.120571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:45.181929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1274/2562] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:10:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:45.195336 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:45.196822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:45.273753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f7942-8d58-401c-af18-56fe55e9ff09 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:45.281504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:45.282151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:45.282448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:45.282721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:45.293037 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:45.294616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366d41fe-e234-49e8-b7bf-00f70c964416 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:45.304179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1289/2563] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:10:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:10:45.633905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:45.635371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:45.704686 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991de0f4-e0a7-41e0-8353-e60aa6e23c11 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:45.709633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:45.710269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:45.710555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:45.710900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] PATH_INFO: `/v3/users/25a9bb79ec58472b82724c2d3c58a74f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:45.712071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25a9bb79ec58472b82724c2d3c58a74f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:45.714104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:46.062659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25a9bb79ec58472b82724c2d3c58a74f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:46.063427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25a9bb79ec58472b82724c2d3c58a74f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:46.064965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:46.072636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25a9bb79ec58472b82724c2d3c58a74f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:46.092388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25a9bb79ec58472b82724c2d3c58a74f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:46.093296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25a9bb79ec58472b82724c2d3c58a74f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:46.248003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1275/2564] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:45 2022] DELETE /identity/v3/users/25a9bb79ec58472b82724c2d3c58a74f => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:46.252426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:46.253097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:46.253097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:46.275715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:46.276980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:46.353576 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1290/2565] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:46.364400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:46.366527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:46.435182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51966dd2-4504-4124-a957-8280d9d45855 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:46.439965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:46.440613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:46.441208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:46.441643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:46.452731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:46.454152 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:46.465613 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1276/2566] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:47.034680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:47.036510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:47.054261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018ffea2-b43f-4672-92e8-8d39a46b2958 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:47.059375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:47.060069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:47.060425 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:47.060699 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] PATH_INFO: `/v3/projects/5ca5bf5c5a264063a4f720507f48a06f` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:47.061954 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ca5bf5c5a264063a4f720507f48a06f)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:47.063856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:47.094677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:47.239589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ca5bf5c5a264063a4f720507f48a06f', 'request_id': 'req-9ede148d-3190-41c3-b08d-e6739b140447'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.246319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ca5bf5c5a264063a4f720507f48a06f', 'request_id': 'req-9ede148d-3190-41c3-b08d-e6739b140447'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.253230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ca5bf5c5a264063a4f720507f48a06f', 'request_id': 'req-9ede148d-3190-41c3-b08d-e6739b140447'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.383955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] The token cache is being invalidate because project 5ca5bf5c5a264063a4f720507f48a06f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:47.384272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.384946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.386552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1291/2567] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:47 2022] DELETE /identity/v3/projects/5ca5bf5c5a264063a4f720507f48a06f => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:47.391114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:47.400991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d0c25c-3b15-4ef0-af3a-8e94e6a60129 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:47.459333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:47.459961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:47.460279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:47.460557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] PATH_INFO: `/v3/users/81ac80f7256e4f7bb8b9ad24c6b61233` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:47.461769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ac80f7256e4f7bb8b9ad24c6b61233)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:47.463728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:47.953645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ac80f7256e4f7bb8b9ad24c6b61233'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.954258 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ac80f7256e4f7bb8b9ad24c6b61233'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.961431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ac80f7256e4f7bb8b9ad24c6b61233'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.973855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ac80f7256e4f7bb8b9ad24c6b61233'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:47.974238 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:47.975188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ac80f7256e4f7bb8b9ad24c6b61233'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.130527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1277/2568] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:47 2022] DELETE /identity/v3/users/81ac80f7256e4f7bb8b9ad24c6b61233 => generated 0 bytes in 740 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:48.367803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:48.369280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:48.383416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ede148d-3190-41c3-b08d-e6739b140447 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:48.463758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:48.464679 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:48.465175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:48.465457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] PATH_INFO: `/v3/users/9b0f802befa642a59860da680ff548a5` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:48.466887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b0f802befa642a59860da680ff548a5)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:48.469750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:48.687985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:48.689550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:48.705894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7960a939-8749-48e6-871f-271bad75689f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:48.711349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:48.712163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:48.712592 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:48.712994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] PATH_INFO: `/v3/projects/6c1962c3350c4a0b9e9abba842223b4e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:48.714411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1962c3350c4a0b9e9abba842223b4e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:48.716412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:48.852815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b0f802befa642a59860da680ff548a5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.853501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b0f802befa642a59860da680ff548a5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.861623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b0f802befa642a59860da680ff548a5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.862383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b0f802befa642a59860da680ff548a5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.876018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9225a4a9-7437-430c-96cf-2b286ad2b57c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b0f802befa642a59860da680ff548a5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.927769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1962c3350c4a0b9e9abba842223b4e', 'request_id': 'req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.934632 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1962c3350c4a0b9e9abba842223b4e', 'request_id': 'req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.941161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1962c3350c4a0b9e9abba842223b4e', 'request_id': 'req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:48.979121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1292/2569] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:48 2022] DELETE /identity/v3/users/9b0f802befa642a59860da680ff548a5 => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:48.985298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:48.985651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:48.985946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.005857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:49.007134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.034406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:49.069210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1293/2570] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:49.079219 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.080905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.083867 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] The token cache is being invalidate because project 6c1962c3350c4a0b9e9abba842223b4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:49.084322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:49.084974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3242c7f1-19f0-4ab6-ac8a-bed3ce2768d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:49.090584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1278/2571] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:48 2022] DELETE /identity/v3/projects/6c1962c3350c4a0b9e9abba842223b4e => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:49.092870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147f69-cd7d-4462-8a3d-64dbb04b96f5 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.098273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147f69-cd7d-4462-8a3d-64dbb04b96f5 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.098273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147f69-cd7d-4462-8a3d-64dbb04b96f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.118997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-72147f69-cd7d-4462-8a3d-64dbb04b96f5 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:49.121514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-72147f69-cd7d-4462-8a3d-64dbb04b96f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.131280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1279/2572] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:49.140839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.141138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.141411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.149507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8c6826-5549-48ff-8812-d2efad8d7e0e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.153703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.154335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.154586 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.154862 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.157212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.170144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.171635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.173803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:49.174963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.183618 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1280/2573] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:49.183852 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1294/2574] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:49.187970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.200732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7cbfe7-0677-47e8-ba7e-ca28fb9e1848 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.206837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.207533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.207823 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.208124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.208825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:49.209269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.210761 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.224977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1281/2575] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:10:49.233286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.244531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717df37-6460-4a8c-85c8-7f314e111a45 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.249439 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.250060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.250355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.250622 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.251266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.253255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.391737 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1295/2576] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/projects => generated 389 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:49.396046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.407016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e7f9e9-acb8-41b4-83f5-33e209d6e8b4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.472852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.472852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.472852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.472852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.473459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.473770 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.602183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1282/2577] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/users => generated 295 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:49.606160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.617146 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185778f9-0f70-461e-8183-f545152ab76d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.622108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.627271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.627271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.627271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.627271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.627271 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.638377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1296/2578] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:49.643244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.653292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017fd364-0e81-4f31-96b4-50e5e518a2cb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.658324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.659043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.659400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.660230 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.660965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.662653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.674961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1283/2579] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:49.686357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.699508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd403895-6877-4c2d-a91e-233759ce5bdb admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.704891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.705348 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.705666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.705906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] PATH_INFO: `/v3/projects/145dd5f4d1a142678b3aaa390cdbdd67/users/1431a80634984ce3b35c281fcbc9fee7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.725235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.727481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.727628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] RBAC: Authorizing `identity:create_grant(project_id=145dd5f4d1a142678b3aaa390cdbdd67, user_id=1431a80634984ce3b35c281fcbc9fee7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.730957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf10c7-2734-4bd0-8aa8-ec24174ac660 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.746394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8358f2aa-a01e-413b-9bce-c33652e9344b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:49.752353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:49.753017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.753300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.753574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] PATH_INFO: `/v3/projects/357cd4132a814121ad5c62c6db670c10` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.754777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] RBAC: Authorizing `identity:delete_project(project_id=357cd4132a814121ad5c62c6db670c10)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:49.756748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:49.783409 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=9,keystone:SELECT=86,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:49.855315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1297/2580] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:49 2022] PUT /identity/v3/projects/145dd5f4d1a142678b3aaa390cdbdd67/users/1431a80634984ce3b35c281fcbc9fee7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:49.868030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:49.868341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:49.868613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:49.900296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] MFA Rules not processed for user `1431a80634984ce3b35c281fcbc9fee7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:49.901511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:49.904767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '357cd4132a814121ad5c62c6db670c10', 'request_id': 'req-73ff517d-406f-4055-b41e-1eb73119f845'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:49.911825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '357cd4132a814121ad5c62c6db670c10', 'request_id': 'req-73ff517d-406f-4055-b41e-1eb73119f845'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:49.917967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '357cd4132a814121ad5c62c6db670c10', 'request_id': 'req-73ff517d-406f-4055-b41e-1eb73119f845'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:49.970046 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1298/2581] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:49.974646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:49.984341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:50.042380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d7c90c-fafa-4db0-a7d4-8ac8166e0dd2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.046776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] The token cache is being invalidate because project 357cd4132a814121ad5c62c6db670c10 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:50.047250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.047956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.048280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.048906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.049180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.049452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.050049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1284/2582] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:49 2022] DELETE /identity/v3/projects/357cd4132a814121ad5c62c6db670c10 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:50.050167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.052028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.053713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.055663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:50.071636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ff517d-406f-4055-b41e-1eb73119f845 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.076684 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.077843 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.078124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.078392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] PATH_INFO: `/v3/users/d5a96c5c41ae4c2fb66f8cf5bf9a3f03` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.079694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5a96c5c41ae4c2fb66f8cf5bf9a3f03)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.081637 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.220984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1299/2583] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:49 2022] POST /identity/v3/projects => generated 389 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:50.224879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.226525 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:50.293194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de24eb4-1c07-4fa9-b840-9702dd7e6789 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.298504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.299104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.299381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.299658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.300323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.302261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.394991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5a96c5c41ae4c2fb66f8cf5bf9a3f03'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.396261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5a96c5c41ae4c2fb66f8cf5bf9a3f03'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.404399 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5a96c5c41ae4c2fb66f8cf5bf9a3f03'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.417544 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5a96c5c41ae4c2fb66f8cf5bf9a3f03'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.418429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5a96c5c41ae4c2fb66f8cf5bf9a3f03'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:50.427260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1300/2584] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:50 2022] POST /identity/v3/users => generated 301 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:50.431350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.433153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:50.446979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d071bd35-6a58-4c23-94ea-7093f152f4cc admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.451668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.452312 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.452668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.452857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.453501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.455114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.465914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1301/2585] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:50.471376 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.481811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a0554-e033-45b1-b981-4bebd37cf5f0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.487058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.487721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.487996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.488315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] PATH_INFO: `/v3/projects/cdc683c636ee4c0fb92e25701ff40858/users/2e6acc87315044eebfcf4336d135674d/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.505063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc683c636ee4c0fb92e25701ff40858, user_id=2e6acc87315044eebfcf4336d135674d, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.507727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.563950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1285/2586] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:50 2022] DELETE /identity/v3/users/d5a96c5c41ae4c2fb66f8cf5bf9a3f03 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:50.641339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1302/2587] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:50 2022] PUT /identity/v3/projects/cdc683c636ee4c0fb92e25701ff40858/users/2e6acc87315044eebfcf4336d135674d/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:50.645772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.711073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce2269a-fb18-4962-9881-5add2b8de0c8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.731799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:50.769803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.770412 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.770822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.771101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.771746 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.773359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95b326-8957-4767-b532-b9ec34a5aa3f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.783402 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1286/2588] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:50.787532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.798507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67805d9a-4b3d-46be-a4c1-975ce218022c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:50.803551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:50.804488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.804840 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.805187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] PATH_INFO: `/v3/projects/cdc683c636ee4c0fb92e25701ff40858/users/2e6acc87315044eebfcf4336d135674d/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.807375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdc683c636ee4c0fb92e25701ff40858, user_id=2e6acc87315044eebfcf4336d135674d, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:50.810725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:50.823904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1303/2589] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:50 2022] PUT /identity/v3/projects/cdc683c636ee4c0fb92e25701ff40858/users/2e6acc87315044eebfcf4336d135674d/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:50.835072 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:50.835365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:50.837787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:50.870181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] MFA Rules not processed for user `2e6acc87315044eebfcf4336d135674d`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:50.871496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:50.951863 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1287/2590] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:50 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:50.963027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:50.968870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:51.007044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:51.030505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f8dc1a-8b6d-4e10-8f32-ecaafe14d20f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:51.034529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:51.035128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:51.035409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:51.035722 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:51.045681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:51.047683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:51.057573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1304/2591] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:50 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:51.146074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:51.147767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:51.227761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:51.260668 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3850783e-ec10-426d-8ff7-8368b9ca8d3f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:51.271883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:51.272811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:51.273665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:51.274612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] PATH_INFO: `/v3/projects/67894b3b17e74bc8985ee5f61fe8b692` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:51.278546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=67894b3b17e74bc8985ee5f61fe8b692)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:51.284051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:51.347903 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a20557-4862-49fb-8a78-911fbc17c3e0 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:51.352966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:51.353605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:51.353887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:51.354158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] PATH_INFO: `/v3/users/1431a80634984ce3b35c281fcbc9fee7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:51.355383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1431a80634984ce3b35c281fcbc9fee7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:51.357436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:51.494633 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:51.496301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67894b3b17e74bc8985ee5f61fe8b692', 'request_id': 'req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.503225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67894b3b17e74bc8985ee5f61fe8b692', 'request_id': 'req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.509041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67894b3b17e74bc8985ee5f61fe8b692', 'request_id': 'req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.637913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] The token cache is being invalidate because project 67894b3b17e74bc8985ee5f61fe8b692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:51.638133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.638800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.640410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1288/2592] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:51 2022] DELETE /identity/v3/projects/67894b3b17e74bc8985ee5f61fe8b692 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:51.645956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:51.647429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:51.723126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d8fe7-8f46-48b9-80d7-476a8e1e57fb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:51.728347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:51.731082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:51.731082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:51.731082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] PATH_INFO: `/v3/users/a8f8f809dc1d4299a234cee2b03eb002` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:51.731082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8f8f809dc1d4299a234cee2b03eb002)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:51.731492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1431a80634984ce3b35c281fcbc9fee7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.732092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1431a80634984ce3b35c281fcbc9fee7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.733150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:51.740993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1431a80634984ce3b35c281fcbc9fee7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.741717 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1431a80634984ce3b35c281fcbc9fee7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.756092 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-696793fd-4ddc-4b68-91d8-363d32b3e7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1431a80634984ce3b35c281fcbc9fee7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:51.891254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1305/2593] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:51 2022] DELETE /identity/v3/users/1431a80634984ce3b35c281fcbc9fee7 => generated 0 bytes in 660 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:51.892666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:51.892953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:51.893223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:51.915125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:51.916431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:51.966000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:51.978171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1306/2594] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:51.988179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:51.989776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:52.051007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389d2b72-404b-4046-a5e1-3d1da3411b11 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:52.054732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:52.055433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:52.055609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:52.055881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:52.066581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:52.068043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:52.078680 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1307/2595] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:52.121810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8f8f809dc1d4299a234cee2b03eb002'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.122557 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8f8f809dc1d4299a234cee2b03eb002'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.131307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8f8f809dc1d4299a234cee2b03eb002'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.144752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8f8f809dc1d4299a234cee2b03eb002'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.145417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8f8f809dc1d4299a234cee2b03eb002'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.246267 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1289/2596] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:51 2022] DELETE /identity/v3/users/a8f8f809dc1d4299a234cee2b03eb002 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:52.571323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:52.572803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:52.586178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94300b05-e0a9-4dce-9575-34b745021889 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:52.663966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:52.664810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:52.665102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:52.665375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] PATH_INFO: `/v3/projects/145dd5f4d1a142678b3aaa390cdbdd67` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:52.666749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=145dd5f4d1a142678b3aaa390cdbdd67)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:52.668757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:52.819116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '145dd5f4d1a142678b3aaa390cdbdd67', 'request_id': 'req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.825854 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '145dd5f4d1a142678b3aaa390cdbdd67', 'request_id': 'req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.831970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '145dd5f4d1a142678b3aaa390cdbdd67', 'request_id': 'req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.934538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:52.936301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:52.951001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0bef7d-4b93-4444-b9d1-b8fe9c42b332 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:52.986792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] The token cache is being invalidate because project 145dd5f4d1a142678b3aaa390cdbdd67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:52.990419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.990419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:52.990419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1308/2597] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:52 2022] DELETE /identity/v3/projects/145dd5f4d1a142678b3aaa390cdbdd67 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:52.993689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:52.995264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:53.013464 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:53.026897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.027815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.028106 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.028406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] PATH_INFO: `/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.029734 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6b8b95133bcf48e387993bf8fd66c4c6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.032183 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:53.064758 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a78ec2-c859-4719-ba69-1bd3a5a6d3e9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:53.070319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.070947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.071247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.071496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] PATH_INFO: `/v3/users/2e6acc87315044eebfcf4336d135674d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.072774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e6acc87315044eebfcf4336d135674d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.074787 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:53.210902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6b8b95133bcf48e387993bf8fd66c4c6', 'request_id': 'req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.217485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6b8b95133bcf48e387993bf8fd66c4c6', 'request_id': 'req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.224454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6b8b95133bcf48e387993bf8fd66c4c6', 'request_id': 'req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.368823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e6acc87315044eebfcf4336d135674d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.369880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e6acc87315044eebfcf4336d135674d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.380105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e6acc87315044eebfcf4336d135674d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.381299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e6acc87315044eebfcf4336d135674d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.403090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e6acc87315044eebfcf4336d135674d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.413237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] The token cache is being invalidate because project 6b8b95133bcf48e387993bf8fd66c4c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:53.413535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.414227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7a7ae4f0-b99c-4cae-9520-87381d44b7c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:53.415969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1290/2598] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:52 2022] DELETE /identity/v3/projects/6b8b95133bcf48e387993bf8fd66c4c6 => generated 0 bytes in 482 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:53.422509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf4a-93e2-4a22-94c2-fd894e1eada8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.422856 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf4a-93e2-4a22-94c2-fd894e1eada8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.423145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf4a-93e2-4a22-94c2-fd894e1eada8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.445039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e715cf4a-93e2-4a22-94c2-fd894e1eada8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:53.446269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e715cf4a-93e2-4a22-94c2-fd894e1eada8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:53.509904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1291/2599] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:53.518385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.526269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.526269 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.526634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1309/2600] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:52 2022] DELETE /identity/v3/users/2e6acc87315044eebfcf4336d135674d => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:53.542873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:53.544104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:53.606782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1292/2601] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:53.611777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:53.613250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:53.631881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7e5d8c-572f-43fe-9239-104ef3382fd5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:53.636883 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.637492 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.637781 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.638055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.638745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:53.639068 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.640748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:53.654166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1310/2602] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:10:53.659612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:53.669470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25cb42a-c7c5-45cc-b740-409a9a4589dc None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:53.674724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.675328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.675603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.675874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.676983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.679125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:53.878301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1293/2603] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:53 2022] POST /identity/v3/projects => generated 365 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:53.883945 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:53.900049 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ba004-97f2-401e-be93-b9736929721b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:53.969842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:53.970530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:53.970812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:53.971084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:53.971727 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:53.973681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.072694 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.074594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.106684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1311/2604] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:53 2022] POST /identity/v3/users => generated 283 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:54.114176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.118099 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.134236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f3fdd0-54b3-4e21-bf74-8472d2eb47fa admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.140675 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.141347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.141629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.141950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] PATH_INFO: `/v3/projects/cdc683c636ee4c0fb92e25701ff40858` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.144108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdc683c636ee4c0fb92e25701ff40858)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.146111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.150717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963ab9ca-6827-467e-af91-f5d0b973c5e4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.161353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.162051 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.162395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.162683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.164629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.179608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:54.232893 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.234336 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.244340 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1294/2605] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:54 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:54.245242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.256818 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0e4a5-f53c-4fe7-a635-13a067c4df3a service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.262462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.263033 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.263320 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.263615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.264324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.266579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.277433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1295/2606] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:54.281600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.291664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d0ac72-bb36-474e-b8e0-b6592131ba09 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.296986 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.297624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.297907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.298185 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.298844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.300534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.316126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1296/2607] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:54.317120 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.328454 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2383b9b6-68e9-461e-b7c0-1957f5f9e769 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.335745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdc683c636ee4c0fb92e25701ff40858', 'request_id': 'req-20aeed82-3d21-4f80-93ee-af638bb0d815'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:54.344320 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdc683c636ee4c0fb92e25701ff40858', 'request_id': 'req-20aeed82-3d21-4f80-93ee-af638bb0d815'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:54.351041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdc683c636ee4c0fb92e25701ff40858', 'request_id': 'req-20aeed82-3d21-4f80-93ee-af638bb0d815'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:54.393349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.397822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.397822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.397822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] PATH_INFO: `/v3/projects/cb8e297c5acb4630afa1855b6b39f474/users/6f1daeb7a60240c3b3d7353ccdf22843/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.409931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb8e297c5acb4630afa1855b6b39f474, user_id=6f1daeb7a60240c3b3d7353ccdf22843, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.412805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182c12f6-11e5-4262-a338-58dd450c4c6d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.486843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] The token cache is being invalidate because project cdc683c636ee4c0fb92e25701ff40858 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-20aeed82-3d21-4f80-93ee-af638bb0d815 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1312/2608] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:54 2022] DELETE /identity/v3/projects/cdc683c636ee4c0fb92e25701ff40858 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c6d5-33fe-461c-9fd6-3cdaf74a206d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c6d5-33fe-461c-9fd6-3cdaf74a206d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.513730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e5c6d5-33fe-461c-9fd6-3cdaf74a206d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.516172 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-45e5c6d5-33fe-461c-9fd6-3cdaf74a206d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:54.517373 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-45e5c6d5-33fe-461c-9fd6-3cdaf74a206d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.525259 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1313/2609] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:54.533111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.533321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.533571 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.553711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:54.554909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.564264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1314/2610] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:54.568653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.579236 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8ef0-28e7-46db-9480-4ab90f10d749 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.579669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1297/2611] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:54 2022] PUT /identity/v3/projects/cb8e297c5acb4630afa1855b6b39f474/users/6f1daeb7a60240c3b3d7353ccdf22843/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:54.605585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.605952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.606148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.640400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] MFA Rules not processed for user `6f1daeb7a60240c3b3d7353ccdf22843`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:54.641634 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.651153 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.651789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.652054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.652366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.653687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:54.653687 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.655286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.672891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1315/2612] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:10:54.674103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.685444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ab613b-589d-4c61-8580-4534ad6a467f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.691430 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.692114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.692447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.692726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.693630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.695674 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.724232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1298/2613] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:54.728640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.730319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:54.745393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5762f46-65c7-406f-b203-e213e04cd1ed None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.750834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.751487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.751797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.752197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.752879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.754717 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.891617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1299/2614] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/projects => generated 367 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:54.892625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1316/2615] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/projects => generated 373 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:54.897141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.899040 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:54.972279 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32733978-027c-4363-af8f-92895a0bba53 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.974827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc507e-aff1-4870-aed5-102af20140d4 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:54.977806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.981546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.981546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.981546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.981546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.982024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:54.982139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:54.982244 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:54.982401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:54.982686 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:54.983366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:54.986105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.110424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1300/2616] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/users => generated 287 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:55.114497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.124784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15902334-7906-4eab-a0ce-c6489fed19ca admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.129513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.130110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.130394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.130667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.131307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.133022 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.143484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1301/2617] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:55.147342 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.157486 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e8bdd2-cdbe-43df-afda-fa76fa33d002 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.162675 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.163256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.163571 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.163849 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.164532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.166125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.173534 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1317/2618] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:54 2022] POST /identity/v3/users => generated 290 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:10:55.176387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.179209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1302/2619] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:55.187387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f4d2d-fe5e-405d-afad-7add55022fff admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.190586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.194659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.196454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.196630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.197277 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.199724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.203035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.209560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c778cb-a5c5-48c8-ba82-8e39970a4a4c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.215493 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.219619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.219619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.219619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] PATH_INFO: `/v3/projects/588b698b09424604b701e64bdea23ba3/users/765e5474fedf45c7969e4bd5d3675b4e/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.236452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1318/2620] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:55.237780 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=588b698b09424604b701e64bdea23ba3, user_id=765e5474fedf45c7969e4bd5d3675b4e, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.238383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.243046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bd8b4e-4019-4d48-874e-181fb84cf443 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.257701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e44e0aa-3317-4bab-b0be-8a059074a3c9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.263849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.264824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.265087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.265372 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] PATH_INFO: `/v3/projects/429953e5444b4f68ac3e45688b3178ef/users/4017666e1bda4453bbc9a7559b376120/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.287540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=429953e5444b4f68ac3e45688b3178ef, user_id=4017666e1bda4453bbc9a7559b376120, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.290894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.336300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1303/2621] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:55 2022] PUT /identity/v3/projects/588b698b09424604b701e64bdea23ba3/users/765e5474fedf45c7969e4bd5d3675b4e/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:55.340679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.340972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.341227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.377969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] MFA Rules not processed for user `765e5474fedf45c7969e4bd5d3675b4e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:55.379208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:55.449368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1304/2622] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:55.461030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1319/2623] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:55 2022] PUT /identity/v3/projects/429953e5444b4f68ac3e45688b3178ef/users/4017666e1bda4453bbc9a7559b376120/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:55.462321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.463744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:55.470206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.527813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7bc783-2203-4df8-a646-7c12ae763bcf None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.542507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809d8ae6-ff86-494b-ae75-58b2222882e7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.542996 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:55.548559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.549171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.550623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.550623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.550623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.552036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.566459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1320/2624] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:10:55.567716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.578947 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639522ef-bc58-44cb-9798-952e38366565 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.584987 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.585723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.590378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.590540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.590540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] PATH_INFO: `/v3/projects/429953e5444b4f68ac3e45688b3178ef/users/4017666e1bda4453bbc9a7559b376120/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.590720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.590720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.590720 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.590976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] RBAC: Authorizing `identity:create_grant(project_id=429953e5444b4f68ac3e45688b3178ef, user_id=4017666e1bda4453bbc9a7559b376120, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.591213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd07ec-bd0f-4700-bb9d-7b91a612929a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.608241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1321/2625] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:55 2022] PUT /identity/v3/projects/429953e5444b4f68ac3e45688b3178ef/users/4017666e1bda4453bbc9a7559b376120/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:55.617043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.617321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.617621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.650139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] MFA Rules not processed for user `4017666e1bda4453bbc9a7559b376120`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:55.651472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:55.661206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.662214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.672810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1305/2626] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:55.696923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.709365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b1e994-a299-4139-bc84-492c2720e447 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.730868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1322/2627] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:55 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:55.743698 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:55.745160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:55.771924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.772772 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.772980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.773237 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] PATH_INFO: `/v3/users/765e5474fedf45c7969e4bd5d3675b4e` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.774440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] RBAC: Authorizing `identity:delete_user(user_id=765e5474fedf45c7969e4bd5d3675b4e)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.777133 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.811075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6c0bf7-7772-49b3-bbe6-38e1516d78b1 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:55.815231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:55.815843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:55.816125 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:55.816429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:55.827656 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:55.829168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:55.839878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1323/2628] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:56.006516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '765e5474fedf45c7969e4bd5d3675b4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:56.007141 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '765e5474fedf45c7969e4bd5d3675b4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:56.016271 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '765e5474fedf45c7969e4bd5d3675b4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:56.028250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '765e5474fedf45c7969e4bd5d3675b4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:56.028973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a650483c-84a6-4fb0-9195-e0a401ad8922 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '765e5474fedf45c7969e4bd5d3675b4e'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:56.138428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:56.162283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:56.172495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09958594-47d1-42a0-94f5-88e2889fe315 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:56.177997 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:56.178632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:56.178910 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:56.179180 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:56.180956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:56.189661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1306/2629] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:10:55 2022] DELETE /identity/v3/users/765e5474fedf45c7969e4bd5d3675b4e => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:10:56.196864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:56.197153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:56.197425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:56.221739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:56.222894 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:56.267763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:56.269366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:56.283935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1324/2630] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:10:56.294916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1307/2631] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:10:56.302315 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:56.303913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:56.377200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b490c739-9e5d-4d0f-9f31-a33011aad78d service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:56.464021 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:56.464670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:56.464955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:56.465232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:56.475909 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:56.477448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf0c5cc-d795-4089-86d7-9a2b6589866a service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:56.489280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1325/2632] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:57.152781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:57.154341 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:57.168004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0fc26b-b041-42ce-abdb-66f6bc822055 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:57.177948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:57.178658 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:57.178970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:57.179250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] PATH_INFO: `/v3/projects/588b698b09424604b701e64bdea23ba3` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:57.180540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=588b698b09424604b701e64bdea23ba3)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:57.182512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:57.288032 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:57.373158 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '588b698b09424604b701e64bdea23ba3', 'request_id': 'req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:57.380995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '588b698b09424604b701e64bdea23ba3', 'request_id': 'req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:57.387564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '588b698b09424604b701e64bdea23ba3', 'request_id': 'req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:57.555485 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] The token cache is being invalidate because project 588b698b09424604b701e64bdea23ba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:10:57.556047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:57.556978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-11dd34e6-70cc-4e25-932f-e0ed9b6c9c83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:10:57.558974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1308/2633] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:10:57 2022] DELETE /identity/v3/projects/588b698b09424604b701e64bdea23ba3 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:57.564518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:57.565022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:57.565140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:57.587812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:57.589104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:57.673929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1326/2634] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:57.682509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:57.682785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:57.683092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:57.708946 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:57.710709 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:57.720114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1309/2635] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:57.724531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:57.735249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c67955-1227-4e2c-99bd-d588547cdef2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:57.742131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:57.743462 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:57.744240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:57.744502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:57.745882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:10:57.753863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:57.753863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:57.770748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1327/2636] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:10:57.773270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:57.785143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630875e-11bf-4fe8-b275-df04b8bd1e88 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:57.791546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:57.792330 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:57.792689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:57.793031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:57.793762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:57.796273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:57.965209 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1310/2637] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:10:57 2022] POST /identity/v3/projects => generated 365 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:57.969938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:57.980853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bfe434-dde7-4572-bfe1-42f29d562dd8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:58.041907 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:58.042391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.042667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.042940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.043563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:58.045480 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:58.163984 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1328/2638] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:10:57 2022] POST /identity/v3/users => generated 283 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:10:58.168379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:58.180030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b2bac-91ee-4f61-89de-afa1b9cc4a03 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:58.184846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:58.185452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.185735 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.186041 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.186690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:58.188353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:58.199001 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1311/2639] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:58.203118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:58.213288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4795c7f6-8992-442f-89a2-ae94456493c6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:58.218582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:58.219224 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.219511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.219786 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.220459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:58.222098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f615cf60-f84f-4047-a526-5c9666748194 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:58.233071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1329/2640] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:10:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:10:58.239846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:58.249444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb19764b-5750-4f94-9b88-860ab0f50ef6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:58.254031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:58.254676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.254955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.255233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] PATH_INFO: `/v3/projects/5659b1f3eb004083b34f4a0907f11dba/users/54ad72f6b9684d36b7f05c80c9360485/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.270404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=5659b1f3eb004083b34f4a0907f11dba, user_id=54ad72f6b9684d36b7f05c80c9360485, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:58.273143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:58.440888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1312/2641] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:10:58 2022] PUT /identity/v3/projects/5659b1f3eb004083b34f4a0907f11dba/users/54ad72f6b9684d36b7f05c80c9360485/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:10:58.465858 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.466476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.466749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.496921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] MFA Rules not processed for user `54ad72f6b9684d36b7f05c80c9360485`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:10:58.498212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:58.563388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1330/2642] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:10:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:10:58.575100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:10:58.577600 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:10:58.635970 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31eba57d-f5f5-4256-8193-37ab3c69e8ba admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:10:58.640188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:10:58.640799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:10:58.641077 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:10:58.641347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:10:58.649722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:10:58.651134 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc498613-1910-468e-a28c-a4d820cddc37 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:10:58.660616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1313/2643] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:10:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:10:59.263483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:10:59.986435 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:01.137457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:01.181120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:01.640854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:02.080721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:04.141928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:04.143356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:04.174299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:04.219415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad42d95-42a8-4e76-865c-e830840ac7d8 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:04.224868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:04.225562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:04.225894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:04.226173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] PATH_INFO: `/v3/users/6f1daeb7a60240c3b3d7353ccdf22843` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:04.227389 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f1daeb7a60240c3b3d7353ccdf22843)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:04.229606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:04.513672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f1daeb7a60240c3b3d7353ccdf22843'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:04.514390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f1daeb7a60240c3b3d7353ccdf22843'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:04.521503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f1daeb7a60240c3b3d7353ccdf22843'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:04.522137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f1daeb7a60240c3b3d7353ccdf22843'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:04.534616 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f1daeb7a60240c3b3d7353ccdf22843'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:04.671955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1331/2644] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:04 2022] DELETE /identity/v3/users/6f1daeb7a60240c3b3d7353ccdf22843 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:11:04.681376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:04.681376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:04.681376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:04.700749 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:04.702016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:04.765431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1314/2645] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:04.775873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:04.777341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:04.838524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb988c-765f-4e95-baa4-b182d5f7747d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:04.842187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:04.849485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:04.849485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:04.849485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:04.853744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:04.855198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:04.865718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1332/2646] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:05.338724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:05.338724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:05.354031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec3d47-4274-45d5-8d63-5f78fb10b0fd None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:05.362830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:05.363909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:05.364455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:05.364955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] PATH_INFO: `/v3/projects/cb8e297c5acb4630afa1855b6b39f474` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:05.366974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb8e297c5acb4630afa1855b6b39f474)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:05.369995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:05.375005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:05.617031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb8e297c5acb4630afa1855b6b39f474', 'request_id': 'req-84304673-d69e-4505-ad94-1431ffd15cd9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:05.629651 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb8e297c5acb4630afa1855b6b39f474', 'request_id': 'req-84304673-d69e-4505-ad94-1431ffd15cd9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:05.635543 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb8e297c5acb4630afa1855b6b39f474', 'request_id': 'req-84304673-d69e-4505-ad94-1431ffd15cd9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:05.798679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] The token cache is being invalidate because project cb8e297c5acb4630afa1855b6b39f474 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:05.799365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:05.799673 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:05.801229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1315/2647] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:05 2022] DELETE /identity/v3/projects/cb8e297c5acb4630afa1855b6b39f474 => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:05.805207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:05.814930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a70148b-a468-4801-be96-c2b7746b73fe service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:05.861159 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:05.874574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:05.875000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:05.875208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:05.875481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] PATH_INFO: `/v3/users/4017666e1bda4453bbc9a7559b376120` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:05.876837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=4017666e1bda4453bbc9a7559b376120)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:05.878772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:06.202799 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4017666e1bda4453bbc9a7559b376120'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:06.203993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4017666e1bda4453bbc9a7559b376120'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:06.218888 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4017666e1bda4453bbc9a7559b376120'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:06.219931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4017666e1bda4453bbc9a7559b376120'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:06.236051 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-b200d151-fb8f-496b-aa3d-5aadfb3864ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4017666e1bda4453bbc9a7559b376120'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:06.329865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1333/2648] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:05 2022] DELETE /identity/v3/users/4017666e1bda4453bbc9a7559b376120 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:06.898358 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:06.899785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:06.914518 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84304673-d69e-4505-ad94-1431ffd15cd9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:06.986204 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:06.986857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:06.987137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:06.987410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] PATH_INFO: `/v3/projects/429953e5444b4f68ac3e45688b3178ef` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:06.988612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=429953e5444b4f68ac3e45688b3178ef)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:06.990845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.143395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '429953e5444b4f68ac3e45688b3178ef', 'request_id': 'req-3a2fe8d9-f131-4bac-a36b-7295697009a0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:07.150249 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '429953e5444b4f68ac3e45688b3178ef', 'request_id': 'req-3a2fe8d9-f131-4bac-a36b-7295697009a0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:07.156430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '429953e5444b4f68ac3e45688b3178ef', 'request_id': 'req-3a2fe8d9-f131-4bac-a36b-7295697009a0'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:07.344576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] The token cache is being invalidate because project 429953e5444b4f68ac3e45688b3178ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:07.350241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:07.350241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3a2fe8d9-f131-4bac-a36b-7295697009a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:07.350241 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1316/2649] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:06 2022] DELETE /identity/v3/projects/429953e5444b4f68ac3e45688b3178ef => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:07.354115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.354611 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.354724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.374941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:07.376253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:07.441026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1334/2650] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:07.454107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.454388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.454662 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.475401 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:07.481293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:07.486017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1317/2651] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:07.490791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.501725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e499f5-4179-4cf4-aa9d-a5cf07a48c86 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.506636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.507246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.507526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.507800 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.508517 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:11:07.508850 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.510515 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.522958 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1335/2652] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:11:07.529065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.538733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78843ea5-9288-4027-b1a7-1e6397716b20 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.543555 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.544111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.544461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.544733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.545882 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.547760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.563902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:07.642188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1318/2653] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:07 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:07.646450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.656884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61112d6-20bc-4cea-ab0a-5b89760e2d64 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.714239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:07.724278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.724855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.725135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.725409 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.726053 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.727982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.824408 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1336/2654] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:07 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:07.828531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.838700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60727b73-4042-40e2-928a-c8abcfa54791 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.845167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.845167 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.845562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.845740 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.846624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.847923 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.863037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1319/2655] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:07.864526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.875019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3bae14-9750-48a2-9d19-6d3dece00b36 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.879826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.880505 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.880790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.881065 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.881743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.883333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f5db3b-f842-4a49-8483-56c7cf598170 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:07.894239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1337/2656] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:07.898505 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:07.909470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b44e10-08b7-46f4-a3c7-988ae732b62c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:07.914030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:07.914670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:07.914943 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:07.915213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] PATH_INFO: `/v3/projects/65da7d02da064eb386a54b25c6dc27f1/users/28342febca8e44b4b7618e84c58801b0/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:07.930804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65da7d02da064eb386a54b25c6dc27f1, user_id=28342febca8e44b4b7618e84c58801b0, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:07.933453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:08.043509 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1320/2657] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:07 2022] PUT /identity/v3/projects/65da7d02da064eb386a54b25c6dc27f1/users/28342febca8e44b4b7618e84c58801b0/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:08.112030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:08.112420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:08.112744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:08.141283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] MFA Rules not processed for user `28342febca8e44b4b7618e84c58801b0`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:08.145979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:08.213045 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1338/2658] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:08.226436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:08.230044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:08.304109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a2c545-fbad-4a9b-8d47-8d51c050fd91 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:08.304109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:08.304109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:08.304109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:08.309338 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:08.313356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:08.314844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab95bba-1cbe-4f6b-a16d-620848e51af5 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:08.325360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1321/2659] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:08.357479 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:08.427057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4368a4-21a9-45a0-b528-40120ef91126 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:08.431849 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:08.432518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:08.432807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:08.433108 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] PATH_INFO: `/v3/users/28342febca8e44b4b7618e84c58801b0` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:08.434272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] RBAC: Authorizing `identity:delete_user(user_id=28342febca8e44b4b7618e84c58801b0)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:08.436212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:08.712391 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28342febca8e44b4b7618e84c58801b0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:08.713063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28342febca8e44b4b7618e84c58801b0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:08.721129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28342febca8e44b4b7618e84c58801b0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:08.721842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28342febca8e44b4b7618e84c58801b0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:08.727513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:08.735551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28342febca8e44b4b7618e84c58801b0'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:08.857057 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1339/2660] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:08 2022] DELETE /identity/v3/users/28342febca8e44b4b7618e84c58801b0 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:08.862188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:08.862470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:08.862739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:08.883243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:08.886507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:08.947562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1322/2661] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:08.957465 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:08.959013 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:09.018500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c03d4d8-c9d8-4cf7-837b-76962288a66a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:09.022187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:09.022817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.023096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.023371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.032346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:09.033765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716868e1-9df5-44e3-9958-04241025a98b service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:09.043255 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1340/2662] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:09.473166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:09.474661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:09.489610 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3986381-edca-4f95-9177-13bd534d92e7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:09.493224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:09.494189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.494564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.494968 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] PATH_INFO: `/v3/projects/65da7d02da064eb386a54b25c6dc27f1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.496451 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65da7d02da064eb386a54b25c6dc27f1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:09.499139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:09.640475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65da7d02da064eb386a54b25c6dc27f1', 'request_id': 'req-b78c8362-d806-4b7e-bc08-02d2b5817da7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:09.646829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65da7d02da064eb386a54b25c6dc27f1', 'request_id': 'req-b78c8362-d806-4b7e-bc08-02d2b5817da7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:09.650429 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=2,keystone:SELECT=93,keystone:DELETE=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:09.652965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65da7d02da064eb386a54b25c6dc27f1', 'request_id': 'req-b78c8362-d806-4b7e-bc08-02d2b5817da7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:09.771168 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] The token cache is being invalidate because project 65da7d02da064eb386a54b25c6dc27f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:09.771465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:09.772205 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-b78c8362-d806-4b7e-bc08-02d2b5817da7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:09.773701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1323/2663] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:09 2022] DELETE /identity/v3/projects/65da7d02da064eb386a54b25c6dc27f1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:09.778734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.779240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.779342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.800944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:09.802233 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:09.866406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1341/2664] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:09.875551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.875937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.876254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.896606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:09.897841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:09.905981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1324/2665] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:09.910306 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:09.920959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84edeeaf-4a9f-42ac-a296-2fc0fc92ef37 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:09.926075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:09.926671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.926950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.927222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.927866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:11:09.928199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:09.929822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:09.943093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1342/2666] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:11:09.947819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:09.957791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12b6cf3-efc7-4e30-981d-c11200de4fc4 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:09.962784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:09.963389 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:09.963664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:09.963958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:09.964627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:09.966515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:10.087853 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1325/2667] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:09 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:10.092050 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:10.103369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ce2043-132d-4d21-aa56-36d0444ad105 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:10.164690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:10.165297 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.165563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.165836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.166466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:10.168424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:10.278198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:10.283915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1343/2668] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:10 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:10.288087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:10.299643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c540a7b9-ab09-4760-9935-61274ebc6ec0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:10.304437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:10.305067 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.305352 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.305625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.306266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:10.307952 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:10.319213 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1326/2669] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:10.325212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:10.336754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d38704d-5fb1-4af9-8788-560afa08e6f6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:10.342996 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:10.343564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.343880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.344269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.344920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:10.346596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6908794d-fdf0-4511-b1e3-97f6ce21f861 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:10.358625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1344/2670] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:10.362887 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:10.373266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1d04f7-44ae-40c0-8a76-fe4686a314bf admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:10.377935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:10.379206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.379487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.379764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] PATH_INFO: `/v3/projects/46f0b6e7755347858190eb6728b2881b/users/9abd5e5b370c4f20a358e425e5004c98/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.395643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=46f0b6e7755347858190eb6728b2881b, user_id=9abd5e5b370c4f20a358e425e5004c98, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:10.398292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:10.497561 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1327/2671] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:10 2022] PUT /identity/v3/projects/46f0b6e7755347858190eb6728b2881b/users/9abd5e5b370c4f20a358e425e5004c98/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:10.510183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.510463 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.511156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.544470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] MFA Rules not processed for user `9abd5e5b370c4f20a358e425e5004c98`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:10.546338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:10.621398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1345/2672] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:10.626725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:10.692788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d653a-99fc-4a4d-919b-36b85bc37c1e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:10.697877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:10.698484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:10.698762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:10.699034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:10.699669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:10.702139 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.005239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1328/2673] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:10 2022] POST /identity/v3/projects => generated 385 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:11.009619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.021290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab670748-a847-4b16-a687-5ce5cd31d802 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.079300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.079905 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.080218 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.080500 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.081121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.083107 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.214764 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1346/2674] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:11 2022] POST /identity/v3/users => generated 293 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:11:11.219444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.230516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de4d27e-65be-476b-9ec9-eb8ba752919e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.235646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.236292 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.236578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.236854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.237500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.239127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.249995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1329/2675] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:11.256178 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.266990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c4aaa-d870-49df-8553-8724690f45d7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.271847 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.272472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.272757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.273034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.273681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.275417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b1040d-b9e8-47fd-9ee3-00594f4a2d16 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.286349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1347/2676] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:11.290754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.301972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dffc1e-57b1-46d6-9ccb-408154f0b95b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.306641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.307299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.307579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.307860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] PATH_INFO: `/v3/projects/f6eb6fb7d96441f590c54f6ea0f38899/users/34f9f358d2624f778ef6b9fee20960b2/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.324091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6eb6fb7d96441f590c54f6ea0f38899, user_id=34f9f358d2624f778ef6b9fee20960b2, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.326864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.434621 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1330/2677] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:11 2022] PUT /identity/v3/projects/f6eb6fb7d96441f590c54f6ea0f38899/users/34f9f358d2624f778ef6b9fee20960b2/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:11.446008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.446288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.446559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.474890 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] MFA Rules not processed for user `34f9f358d2624f778ef6b9fee20960b2`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:11.476136 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:11.521300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:11.541667 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1348/2678] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:11 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:11.546802 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.584478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:11.615164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd308e-a1e4-4cf9-8fbe-af0df9c21f1d admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.619805 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.620466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.620806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.621087 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.621729 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.623671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.734481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1331/2679] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:11 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:11.738681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.749859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7586b9f2-f42e-4958-b6e8-d32deb42d18a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.808123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.808661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.808940 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.809205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.809848 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.812424 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.917303 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1349/2680] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:11 2022] POST /identity/v3/users => generated 299 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:11.922044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.932535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481cd94d-7dbf-43da-9402-1c4166b6c980 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.938239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.938835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.939109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.939377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.940046 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.941679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:11.952229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1332/2681] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:11.956472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:11.966540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe39164-2f16-4f60-9022-8fea75898717 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:11.971228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:11.971867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:11.972170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:11.972452 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] PATH_INFO: `/v3/projects/97d3209524064e14b803b106d6335385/users/a45492d65eda448997d30503b75e852f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:11.988715 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d3209524064e14b803b106d6335385, user_id=a45492d65eda448997d30503b75e852f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:11.991357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:12.080870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1350/2682] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:11 2022] PUT /identity/v3/projects/97d3209524064e14b803b106d6335385/users/a45492d65eda448997d30503b75e852f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:12.084910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:12.095771 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3762ec38-864c-4aae-9de3-8c9ab4f13c89 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:12.153748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:12.154351 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:12.154663 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:12.155487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:12.156122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:12.157755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54045a45-af4e-499c-b776-7f87b0e10369 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:12.168660 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1333/2683] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:12.173660 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:12.185390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed50b516-2fb5-4565-989b-96f652e6372b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:12.190349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:12.191004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:12.191281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:12.191556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] PATH_INFO: `/v3/projects/97d3209524064e14b803b106d6335385/users/a45492d65eda448997d30503b75e852f/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:12.193763 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d3209524064e14b803b106d6335385, user_id=a45492d65eda448997d30503b75e852f, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:12.197097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:12.208442 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1351/2684] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:12 2022] PUT /identity/v3/projects/97d3209524064e14b803b106d6335385/users/a45492d65eda448997d30503b75e852f/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:12.219532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:12.219906 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:12.220299 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:12.250706 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] MFA Rules not processed for user `a45492d65eda448997d30503b75e852f`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:12.251965 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:12.326466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1334/2685] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:12 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:12.338614 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:12.340459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:12.407196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864727ee-e2a4-43d6-9be7-458f360281a3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:12.410990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:12.411642 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:12.411904 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:12.412205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:12.468547 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:12.472135 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:12.473606 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:12.483026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1352/2686] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:12 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:14.706700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:14.717482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94d75cf-126d-46e8-baed-2eb22b910845 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:14.721359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:14.721992 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:14.722284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:14.722562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:14.747974 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:14.783282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:14.784741 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:14.794164 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1335/2687] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:16.865094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:16.874436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf5ce3d-aa4d-4d9e-ab77-0dd1ef2eca9b service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:16.878359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:16.878960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:16.879266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:16.879538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:16.889365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:16.890792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8069c4d5-488d-40aa-bb16-b250cde4d1f0 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:16.902256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1353/2688] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:16 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:22.053045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:22.126406 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5811f70-78b3-470c-be5c-96edd2ced48d service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:22.132343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:22.133148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:22.133495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:22.133781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] PATH_INFO: `/v3/users/9abd5e5b370c4f20a358e425e5004c98` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:22.134995 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9abd5e5b370c4f20a358e425e5004c98)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:22.136955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:22.446364 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9abd5e5b370c4f20a358e425e5004c98'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:22.447014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9abd5e5b370c4f20a358e425e5004c98'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:22.454959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9abd5e5b370c4f20a358e425e5004c98'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:22.469549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9abd5e5b370c4f20a358e425e5004c98'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:22.470332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9abd5e5b370c4f20a358e425e5004c98'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:22.569316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1336/2689] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:22 2022] DELETE /identity/v3/users/9abd5e5b370c4f20a358e425e5004c98 => generated 0 bytes in 517 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:22.574063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f38133-2095-4310-931a-3003ba29af50 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:22.579179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f38133-2095-4310-931a-3003ba29af50 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:22.579179 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f38133-2095-4310-931a-3003ba29af50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:22.595701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-06f38133-2095-4310-931a-3003ba29af50 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:22.597144 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-06f38133-2095-4310-931a-3003ba29af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:22.666856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1354/2690] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:22.677670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:22.679239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:22.735028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:22.746553 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06688446-9aef-4c35-9457-bfe6d9f4a4c6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:22.750872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:22.751488 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:22.751767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:22.752039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:22.762529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:22.764643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:22.775859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1337/2691] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:23.287064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f38133-2095-4310-931a-3003ba29af50 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:23.288842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-06f38133-2095-4310-931a-3003ba29af50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:23.324026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f38133-2095-4310-931a-3003ba29af50 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:23.330043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:23.330735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:23.331137 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:23.331308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] PATH_INFO: `/v3/projects/46f0b6e7755347858190eb6728b2881b` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:23.333142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] RBAC: Authorizing `identity:delete_project(project_id=46f0b6e7755347858190eb6728b2881b)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:23.335101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:23.517350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46f0b6e7755347858190eb6728b2881b', 'request_id': 'req-2f48f07e-bda5-4092-b3de-ec3d8479c62d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:23.523860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46f0b6e7755347858190eb6728b2881b', 'request_id': 'req-2f48f07e-bda5-4092-b3de-ec3d8479c62d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:23.530552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46f0b6e7755347858190eb6728b2881b', 'request_id': 'req-2f48f07e-bda5-4092-b3de-ec3d8479c62d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:23.674096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] The token cache is being invalidate because project 46f0b6e7755347858190eb6728b2881b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:23.674791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:23.675015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:23.676553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1355/2692] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:23 2022] DELETE /identity/v3/projects/46f0b6e7755347858190eb6728b2881b => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:23.680785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:23.690608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c69539-28b5-4f4a-a0c0-e106f39998f6 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:23.756839 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:23.757517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:23.757790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:23.758069 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] PATH_INFO: `/v3/users/34f9f358d2624f778ef6b9fee20960b2` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:23.759348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=34f9f358d2624f778ef6b9fee20960b2)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:23.761331 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:24.021834 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '34f9f358d2624f778ef6b9fee20960b2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:24.022593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '34f9f358d2624f778ef6b9fee20960b2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:24.030393 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '34f9f358d2624f778ef6b9fee20960b2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:24.042541 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '34f9f358d2624f778ef6b9fee20960b2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:24.043363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '34f9f358d2624f778ef6b9fee20960b2'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:24.184949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1338/2693] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:23 2022] DELETE /identity/v3/users/34f9f358d2624f778ef6b9fee20960b2 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:24.761158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:24.762919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:24.777830 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48f07e-bda5-4092-b3de-ec3d8479c62d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:24.844347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:24.844977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:24.845257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:24.845529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] PATH_INFO: `/v3/projects/f6eb6fb7d96441f590c54f6ea0f38899` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:24.846645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6eb6fb7d96441f590c54f6ea0f38899)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:24.848582 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:24.854926 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:25.037283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6eb6fb7d96441f590c54f6ea0f38899', 'request_id': 'req-3018b434-69ea-448b-9b40-fc301d9091a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.044267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6eb6fb7d96441f590c54f6ea0f38899', 'request_id': 'req-3018b434-69ea-448b-9b40-fc301d9091a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.049950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6eb6fb7d96441f590c54f6ea0f38899', 'request_id': 'req-3018b434-69ea-448b-9b40-fc301d9091a7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.173985 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] The token cache is being invalidate because project f6eb6fb7d96441f590c54f6ea0f38899 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:25.174476 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.175145 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.176701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1356/2694] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:24 2022] DELETE /identity/v3/projects/f6eb6fb7d96441f590c54f6ea0f38899 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:25.181026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:25.182615 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:25.197568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcc183-285a-44f3-a585-dbd2ea6bfc5e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:25.264936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:25.265578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:25.265862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:25.266166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] PATH_INFO: `/v3/users/a45492d65eda448997d30503b75e852f` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:25.267359 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a45492d65eda448997d30503b75e852f)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:25.270736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:25.277424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:25.674421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a45492d65eda448997d30503b75e852f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.675015 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a45492d65eda448997d30503b75e852f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.682418 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a45492d65eda448997d30503b75e852f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.694666 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a45492d65eda448997d30503b75e852f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.695500 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fd5a8a7e-09f8-41ac-9742-45b8ad92b992 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a45492d65eda448997d30503b75e852f'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:25.870108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1339/2695] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:25 2022] DELETE /identity/v3/users/a45492d65eda448997d30503b75e852f => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:26.390316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.391823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.406063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3018b434-69ea-448b-9b40-fc301d9091a7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.469350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.469990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.470283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.470561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] PATH_INFO: `/v3/projects/97d3209524064e14b803b106d6335385` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.471776 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d3209524064e14b803b106d6335385)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.473741 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.642481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d3209524064e14b803b106d6335385', 'request_id': 'req-c3107f03-7a67-4187-9756-f8faeae1d214'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.648967 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d3209524064e14b803b106d6335385', 'request_id': 'req-c3107f03-7a67-4187-9756-f8faeae1d214'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.654789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d3209524064e14b803b106d6335385', 'request_id': 'req-c3107f03-7a67-4187-9756-f8faeae1d214'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.765381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] The token cache is being invalidate because project 97d3209524064e14b803b106d6335385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:26.765592 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.766215 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c3107f03-7a67-4187-9756-f8faeae1d214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:26.767702 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1357/2696] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:26 2022] DELETE /identity/v3/projects/97d3209524064e14b803b106d6335385 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:26.773059 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd419-1db3-4517-baad-53d66088881f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.773551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd419-1db3-4517-baad-53d66088881f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.775778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147cd419-1db3-4517-baad-53d66088881f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.794467 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-147cd419-1db3-4517-baad-53d66088881f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:26.795715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-147cd419-1db3-4517-baad-53d66088881f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.864434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1340/2697] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:11:26.873494 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.873823 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.874041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.906384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:26.906384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:26.909433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1358/2698] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:26.913311 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cd419-1db3-4517-baad-53d66088881f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.923751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147cd419-1db3-4517-baad-53d66088881f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.931757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.931757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.931757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.931757 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.932259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:11:26.932259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.932448 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.946021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1341/2699] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:11:26.950623 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:26.961380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb4d4f7-4b81-49fe-8801-451e19b5ba74 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:26.967200 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:26.967873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:26.968268 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:26.969313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:26.969313 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:26.971197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:26.988960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:27.075283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1359/2700] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:26 2022] POST /identity/v3/projects => generated 373 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:27.079947 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.092251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031b6655-85bf-417e-ba28-c73fa00d21e4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.160025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.160733 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.161024 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.161300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.161953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.163888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.300003 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1342/2701] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:27 2022] POST /identity/v3/users => generated 287 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:27.304131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.315600 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698dd1eb-a497-43db-9231-ca34b6d621fa admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.320581 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.321194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.321514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.321770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.322418 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.328206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.348250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1360/2702] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:27.349612 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.355289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:27.370313 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4386303a-1578-4b5d-b9aa-9d3057e8cb73 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.377403 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.378004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.378288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.378562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.379431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.381063 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d7158f-9b83-441c-812f-88d412cf0740 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.394879 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1343/2703] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:27.399202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.416208 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4f2c14-db01-4cc9-9897-bcba90eaefcf admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.421503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.422207 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.422569 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.422932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] PATH_INFO: `/v3/projects/573d91123526408f88c0a9d7612da26c/users/68332caf77ed443ea4828e643ff28f85/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.439121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=573d91123526408f88c0a9d7612da26c, user_id=68332caf77ed443ea4828e643ff28f85, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.442082 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.545801 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1361/2704] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:27 2022] PUT /identity/v3/projects/573d91123526408f88c0a9d7612da26c/users/68332caf77ed443ea4828e643ff28f85/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:27.550987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb2f1-a2f1-4589-80a9-464b98354762 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.551256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb2f1-a2f1-4589-80a9-464b98354762 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.551538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749fb2f1-a2f1-4589-80a9-464b98354762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.573793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-749fb2f1-a2f1-4589-80a9-464b98354762 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:27.575291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-749fb2f1-a2f1-4589-80a9-464b98354762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:27.644086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1344/2705] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:27.653832 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:27.656274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:27.718523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4733b8-691c-4dc2-9937-a50466218e83 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:27.722189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:27.722789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:27.723163 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:27.723441 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:27.733358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:27.734826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:27.745024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1362/2706] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:29.452091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac0f-7939-44fe-abb6-f3ea979d42a7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:29.452669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac0f-7939-44fe-abb6-f3ea979d42a7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:29.452858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63afac0f-7939-44fe-abb6-f3ea979d42a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:29.486915 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-63afac0f-7939-44fe-abb6-f3ea979d42a7 None None] MFA Rules not processed for user `68332caf77ed443ea4828e643ff28f85`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:29.489250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-63afac0f-7939-44fe-abb6-f3ea979d42a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:29.581736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1345/2707] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:29.593173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:29.594872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:29.675028 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c88dbd3-c24d-4238-90c7-e6336305e91c service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:29.685881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:29.685881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:29.685881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:29.685881 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:29.690608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:29.692007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:29.710756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1363/2708] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:29 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:31.791781 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47e2d-83e3-46b0-a402-98a66354afd1 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:31.792353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47e2d-83e3-46b0-a402-98a66354afd1 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:31.792470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad47e2d-83e3-46b0-a402-98a66354afd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:31.815391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7ad47e2d-83e3-46b0-a402-98a66354afd1 None None] MFA Rules not processed for user `68332caf77ed443ea4828e643ff28f85`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:31.816565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7ad47e2d-83e3-46b0-a402-98a66354afd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:31.824785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1346/2709] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:39.393838 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:39.395385 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:39.409150 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59734079-4a7c-4aa0-b69d-f7d01b6f8691 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:39.413014 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:39.413666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:39.413972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:39.414250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:39.415993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:39.482915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:39.498834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:39.500412 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:39.512181 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1364/2710] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:40.618855 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc801-a00a-4614-b698-737909efa04b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:40.619400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc801-a00a-4614-b698-737909efa04b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:40.619400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbc801-a00a-4614-b698-737909efa04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:40.641379 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-78bbc801-a00a-4614-b698-737909efa04b None None] MFA Rules not processed for user `f5e920d3b3044859b767add41690043c`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:40.642581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-78bbc801-a00a-4614-b698-737909efa04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:40.699648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1347/2711] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:43.884008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:43.884008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:43.897633 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96339bfd-90da-4991-bc60-e40d913d54dc service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:43.902553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:43.903228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:43.903531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:43.903807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] PATH_INFO: `/v3/users/54ad72f6b9684d36b7f05c80c9360485` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:43.905033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54ad72f6b9684d36b7f05c80c9360485)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:43.906943 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:44.191035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54ad72f6b9684d36b7f05c80c9360485'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:44.191710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54ad72f6b9684d36b7f05c80c9360485'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:44.200617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54ad72f6b9684d36b7f05c80c9360485'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:44.201524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54ad72f6b9684d36b7f05c80c9360485'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:44.220692 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54ad72f6b9684d36b7f05c80c9360485'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:44.332460 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1365/2712] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:11:43 2022] DELETE /identity/v3/users/54ad72f6b9684d36b7f05c80c9360485 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:44.337082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07842326-135e-4885-8f42-20c124930790 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:44.342495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07842326-135e-4885-8f42-20c124930790 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:44.342495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07842326-135e-4885-8f42-20c124930790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:44.357750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-07842326-135e-4885-8f42-20c124930790 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:44.358983 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-07842326-135e-4885-8f42-20c124930790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:44.376819 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:44.415965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:44.417526 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:44.431908 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1348/2713] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:44.444073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07842326-135e-4885-8f42-20c124930790 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:44.445880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-07842326-135e-4885-8f42-20c124930790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:44.487681 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14325e-787c-4e7e-8fc5-5b11e8270453 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:44.491489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:44.496561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:44.496561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:44.496561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:44.496561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:44.509760 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07842326-135e-4885-8f42-20c124930790 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:44.514108 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:44.514748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:44.515027 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:44.515301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:44.530045 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:44.531166 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:44.542416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1349/2714] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:44.585170 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:44.587375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9020da-4bfc-4389-b775-9e8c96a5731f service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:44.600746 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1366/2715] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:45.385385 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:45.386926 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:45.400368 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661de1af-837d-43d2-b2d0-0f2e080b53e2 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:45.406913 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:45.407582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:45.407895 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:45.408234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] PATH_INFO: `/v3/projects/5659b1f3eb004083b34f4a0907f11dba` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:45.409400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5659b1f3eb004083b34f4a0907f11dba)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:45.411301 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:45.602260 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5659b1f3eb004083b34f4a0907f11dba', 'request_id': 'req-a806c28d-6fdd-455b-bd8b-de1d9da546b1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:45.610199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5659b1f3eb004083b34f4a0907f11dba', 'request_id': 'req-a806c28d-6fdd-455b-bd8b-de1d9da546b1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:45.616680 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5659b1f3eb004083b34f4a0907f11dba', 'request_id': 'req-a806c28d-6fdd-455b-bd8b-de1d9da546b1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:45.730071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] The token cache is being invalidate because project 5659b1f3eb004083b34f4a0907f11dba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:11:45.730380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:45.734043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a806c28d-6fdd-455b-bd8b-de1d9da546b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:11:45.734043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1350/2716] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:11:45 2022] DELETE /identity/v3/projects/5659b1f3eb004083b34f4a0907f11dba => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:45.738415 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:45.738932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:45.739043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:45.748539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:45.760457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:45.761694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:45.828651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1367/2717] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:11:45.836283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:45.836579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:45.836845 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:45.862263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:45.864082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:45.883489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1351/2718] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:45.884710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:45.894915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcd5cb-91c4-4374-a22e-3ec3fa3ef38d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:45.899950 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:45.900891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:45.901243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:45.901510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] PATH_INFO: `/v3/domains` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:45.902402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88422) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:11:45.902766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:45.904554 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:45.932314 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1368/2719] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:11:45.934506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:45.949100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd68c006-edf2-4022-9ee2-5d8937339535 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:45.956192 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:45.956905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:45.958157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:45.958573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:45.959400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:45.962944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.144181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1352/2720] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:11:45 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:46.145676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.157968 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af729556-1b67-4764-8d0f-3cd1ae4c7e30 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.243942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.243942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.243942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.243942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.244808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.244808 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.333934 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1369/2721] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:11:46 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:11:46.337381 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.351334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a88596-7e30-4afa-9ef6-63a3504f29fb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.357812 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.358808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.358808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.359146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.359601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.361250 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.376806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1353/2722] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:46.381022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.391755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc5c25-38c9-4252-b2de-2ed2afbdb952 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.396869 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.403730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.403730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.403730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.403730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.403730 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ee4f45-001f-4936-85b0-45ff7bf6ebee admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.420928 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1370/2723] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:11:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:11:46.422257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.433936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f1629b-2fa4-47d2-9b4b-1c7af144a203 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.439482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.440377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.450026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.450026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] PATH_INFO: `/v3/projects/d3cad599cd5943e8b545de7bacca6fe1/users/ae8540a6f24149abbab456ca1c6b33c7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.450026 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:46.461349 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3cad599cd5943e8b545de7bacca6fe1, user_id=ae8540a6f24149abbab456ca1c6b33c7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.464132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.587993 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1354/2724] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:11:46 2022] PUT /identity/v3/projects/d3cad599cd5943e8b545de7bacca6fe1/users/ae8540a6f24149abbab456ca1c6b33c7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:11:46.593558 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317172-8f7d-4c5a-8269-a9ca2cb553fe None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.593837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317172-8f7d-4c5a-8269-a9ca2cb553fe None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.594078 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d317172-8f7d-4c5a-8269-a9ca2cb553fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.618257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2d317172-8f7d-4c5a-8269-a9ca2cb553fe None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:46.619944 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2d317172-8f7d-4c5a-8269-a9ca2cb553fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:46.697102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1371/2725] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:11:46.704724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:46.706730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:46.774319 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24111dbd-b6e5-4cb0-985d-ac0e33258974 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:46.778282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:46.779764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:46.779921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:46.780264 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:46.790713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:46.792515 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:46.804101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1355/2726] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:11:48.931113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb7f1c-3fe2-46aa-ba3e-0c0b096b9990 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:48.931700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb7f1c-3fe2-46aa-ba3e-0c0b096b9990 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:48.931700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfb7f1c-3fe2-46aa-ba3e-0c0b096b9990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:48.956337 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:11:48.967114 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-2bfb7f1c-3fe2-46aa-ba3e-0c0b096b9990 None None] MFA Rules not processed for user `ae8540a6f24149abbab456ca1c6b33c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:48.968417 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-2bfb7f1c-3fe2-46aa-ba3e-0c0b096b9990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:49.043466 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1372/2727] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:49.055921 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:11:49.057386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:49.119220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3740e32f-5020-47b4-bdfc-607ca5966997 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:11:49.127396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:11:49.128074 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:49.128360 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:49.128638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:49.138642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:11:49.140048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1fc0d-9872-4a4b-af70-794b11ce8145 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:11:49.149948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1356/2728] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:11:49 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:11:51.195935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7511caf-7d86-4320-8ee9-4101256b2a8d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:51.196649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7511caf-7d86-4320-8ee9-4101256b2a8d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:51.196649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7511caf-7d86-4320-8ee9-4101256b2a8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:51.217266 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b7511caf-7d86-4320-8ee9-4101256b2a8d None None] MFA Rules not processed for user `ae8540a6f24149abbab456ca1c6b33c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:51.220232 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b7511caf-7d86-4320-8ee9-4101256b2a8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:51.227610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1373/2729] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:11:55.223876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:11:55.224420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:11:55.224538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:11:55.243966 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] MFA Rules not processed for user `68332caf77ed443ea4828e643ff28f85`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:11:55.245290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:11:55.302700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1357/2730] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:11:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:01.210797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=30 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:05.295419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:26.633142 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:26.633689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:26.633689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:26.684724 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:26.685941 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:26.754568 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1374/2731] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:12:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:26.973057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:26.974476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:27.072334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f71210f-0cad-4d34-87ae-dd4c2045721e None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:27.078274 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:27.078883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:27.079214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:27.079519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:27.088532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:27.090653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:27.098931 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1358/2732] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:12:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:12:36.751757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=40 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:37.095594 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=41 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:37.503396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:37.505081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:37.786386 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c56c6-f91d-4a17-af70-94b9fe2501ee None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:37.799682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:37.804287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:37.804287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:37.804287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:37.805562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:37.823655 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:37.826042 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514a5c6-8de7-4f11-bfd2-6da2f79639cb service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:37.845498 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1375/2733] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:37 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:12:37.962685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:37.964814 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:37.990391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee03c97-c4fa-41e5-b4fa-98192e873a55 service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:37.994832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:37.995790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:37.996499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:37.997343 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:38.001576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:38.015240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:38.017420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:38.030524 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1359/2734] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:12:47.528301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:47.529157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:47.529332 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:47.549335 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:47.566587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:47.566587 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1376/2735] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:12:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:47.571901 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:47.584180 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d76c78-de4e-4d4a-b2c2-709c6aa03fff service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:47.590031 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:47.590627 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:47.590907 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:47.591189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:47.599397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:47.600991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:47.609875 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1360/2736] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:12:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:12:53.520856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:53.522354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:53.602795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1b7ae-f11f-43c5-b231-a93cb1aff57a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:53.609156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:53.609976 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:53.610166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:53.610367 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:53.612440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:53.628721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:53.630618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:53.643392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1377/2737] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:12:53.730382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:53.730382 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:53.744799 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a244-7e6d-4c63-a108-e015c3f589dc service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:53.750196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:53.750837 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:53.751115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:53.751388 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] PATH_INFO: `/v3/users/68332caf77ed443ea4828e643ff28f85` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:53.752538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] RBAC: Authorizing `identity:delete_user(user_id=68332caf77ed443ea4828e643ff28f85)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:53.754473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:53.940303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68332caf77ed443ea4828e643ff28f85'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:53.940973 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68332caf77ed443ea4828e643ff28f85'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:53.949007 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68332caf77ed443ea4828e643ff28f85'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:53.962704 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68332caf77ed443ea4828e643ff28f85'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:53.963459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-51576122-88b7-47cc-9798-2f17ced2fdda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68332caf77ed443ea4828e643ff28f85'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.048103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1361/2738] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:12:53 2022] DELETE /identity/v3/users/68332caf77ed443ea4828e643ff28f85 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:12:54.360919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:54.374187 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b09fb9-fd9a-4f86-b12a-b0decae2bbd7 service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:54.431564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:54.432247 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:54.433043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:54.433596 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] PATH_INFO: `/v3/projects/573d91123526408f88c0a9d7612da26c` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:54.434471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=573d91123526408f88c0a9d7612da26c)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:54.436357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:54.449507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:54.622874 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '573d91123526408f88c0a9d7612da26c', 'request_id': 'req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.629863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '573d91123526408f88c0a9d7612da26c', 'request_id': 'req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.636999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '573d91123526408f88c0a9d7612da26c', 'request_id': 'req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.808349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] The token cache is being invalidate because project 573d91123526408f88c0a9d7612da26c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:12:54.808639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.809263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cbe8c65d-64cb-4380-8ac3-8143c661cf1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:54.810777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1378/2739] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:12:54 2022] DELETE /identity/v3/projects/573d91123526408f88c0a9d7612da26c => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:12:54.815778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:54.816345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:54.816345 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:54.836945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:54.838084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:54.901991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1362/2740] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:54.910390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:54.910696 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:54.910966 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:54.931601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:54.933457 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:54.941356 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1379/2741] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:54.946287 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:54.956511 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e951a7-e915-40c2-bba8-30a0f2fe2380 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:54.962300 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:54.962937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:54.963215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:54.963562 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:54.964685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:12:54.964989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:54.966624 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:54.978822 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1363/2742] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:12:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:12:54.983705 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:54.993263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d8e925-f7cf-4643-9d11-ad13e0fda6e8 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:54.997972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:54.998562 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:54.998842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:54.999113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:54.999771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.001779 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.181098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1380/2743] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:12:54 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:12:55.185433 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:55.196222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b921973-2880-4122-b2d0-ef977968da2b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:55.229288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:55.260281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:55.260281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.260281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.260281 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.260767 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.261128 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.403150 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1364/2744] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:12:55 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:12:55.407450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:55.420064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fffb048-1a24-4659-b797-b80f22c45122 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:55.424899 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:55.425574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.425953 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.426300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.427011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.428863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.439401 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1381/2745] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:12:55.443458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:55.454670 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc65f78f-b3de-407e-933b-801af29a46cc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:55.459405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:55.460489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.460489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.460739 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.461290 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.462927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09f2619-272d-44d3-89d7-51ef5ee718bf admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.473727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1365/2746] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:12:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:12:55.478034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:55.488433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd966e-0a36-4ced-b384-d0ea2bdbac5a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:55.492991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:55.493636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.493913 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.494183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] PATH_INFO: `/v3/projects/393f2656b2ca49e7bf5db6a84ca94891/users/6f5bf76c39dd4c81a2c07e67490dec12/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.508826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] RBAC: Authorizing `identity:create_grant(project_id=393f2656b2ca49e7bf5db6a84ca94891, user_id=6f5bf76c39dd4c81a2c07e67490dec12, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.512029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.676707 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1382/2747] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:12:55 2022] PUT /identity/v3/projects/393f2656b2ca49e7bf5db6a84ca94891/users/6f5bf76c39dd4c81a2c07e67490dec12/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:12:55.687449 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.688036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.688367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.719508 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] MFA Rules not processed for user `6f5bf76c39dd4c81a2c07e67490dec12`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:55.720876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:55.792407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1366/2748] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:12:55 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:12:55.803806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:55.805648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:55.864557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca099c5-e4f8-456c-8ea0-dcae04bede3e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:55.868294 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:55.868938 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:55.869221 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:55.869496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:55.878744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:55.880296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a01ee53-6095-4138-94a6-2760e718333f service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:55.890711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1383/2749] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:12:59.219436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:12:59.289519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac575b86-f171-4ecd-9d4e-8385854889c8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:12:59.296745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:12:59.297453 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:59.297736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:59.298019 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] PATH_INFO: `/v3/users/6f5bf76c39dd4c81a2c07e67490dec12` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:59.299163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f5bf76c39dd4c81a2c07e67490dec12)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:12:59.301283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:12:59.709990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:12:59.712665 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f5bf76c39dd4c81a2c07e67490dec12'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:59.713315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f5bf76c39dd4c81a2c07e67490dec12'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:59.721016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f5bf76c39dd4c81a2c07e67490dec12'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:59.733786 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f5bf76c39dd4c81a2c07e67490dec12'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:59.734495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f5bf76c39dd4c81a2c07e67490dec12'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:12:59.910914 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1367/2750] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:12:59 2022] DELETE /identity/v3/users/6f5bf76c39dd4c81a2c07e67490dec12 => generated 0 bytes in 692 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:12:59.915440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:12:59.915911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:12:59.916031 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:12:59.934733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:12:59.935914 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:12:59.974342 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:00.011509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1384/2751] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:12:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:00.021689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:00.023147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:00.087115 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3450d2f-e532-4e44-a51d-72dbee7e9a40 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:00.091869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:00.092548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:00.092836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:00.093114 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:00.103321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:00.104829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:00.115989 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1368/2752] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:00.718052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:00.722937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:00.736071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629cbdfb-550d-478d-9010-d95572dcc4ca None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:00.741347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:00.742549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:00.742828 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:00.743272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] PATH_INFO: `/v3/projects/393f2656b2ca49e7bf5db6a84ca94891` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:00.744509 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] RBAC: Authorizing `identity:delete_project(project_id=393f2656b2ca49e7bf5db6a84ca94891)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:00.747035 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:00.928673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '393f2656b2ca49e7bf5db6a84ca94891', 'request_id': 'req-890807e0-96b7-4f05-a573-6da5dbf9743d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:00.935523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '393f2656b2ca49e7bf5db6a84ca94891', 'request_id': 'req-890807e0-96b7-4f05-a573-6da5dbf9743d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:00.941054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '393f2656b2ca49e7bf5db6a84ca94891', 'request_id': 'req-890807e0-96b7-4f05-a573-6da5dbf9743d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:01.142323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] The token cache is being invalidate because project 393f2656b2ca49e7bf5db6a84ca94891 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:13:01.142323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:01.142323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:01.142323 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1385/2753] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:13:00 2022] DELETE /identity/v3/projects/393f2656b2ca49e7bf5db6a84ca94891 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:13:02.782790 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:02.791944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e3af91-35d8-4b76-9a00-79908b6c517e service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:02.844065 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:02.848948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:02.848948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:02.848948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:02.848948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:02.922242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:02.923676 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a751de7-b9ea-4650-94a5-68edaf705a8d service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:02.934516 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1369/2754] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:06.895570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:06.897094 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:06.976738 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890807e0-96b7-4f05-a573-6da5dbf9743d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:06.988345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:06.988345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:06.988345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:06.988345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:06.988761 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:06.998700 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:07.001557 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:07.016343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1386/2755] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:10.672034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:10.672583 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:10.672685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:10.694653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:13:10.696197 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:10.705804 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:10.761811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1370/2756] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:13:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:10.775513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:10.778098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:10.835578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:10.855291 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3e7074-3dea-44cc-9c7a-9dd53c2e4c3c service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:10.860402 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:10.861158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:10.861438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:10.861710 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:10.871169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:10.872771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:10.881173 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1387/2757] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:13:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:13:11.765702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:11.765702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:11.787956 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bed254b-e26e-4afe-8e01-d16404c57cf0 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:11.794370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:11.799876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:11.799876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:11.799876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] PATH_INFO: `/v3/users/26aa79753d6e4e49ad66cc32ba786fe8` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:11.799876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=26aa79753d6e4e49ad66cc32ba786fe8)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:11.799876 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:12.122641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26aa79753d6e4e49ad66cc32ba786fe8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.123334 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26aa79753d6e4e49ad66cc32ba786fe8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.130938 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26aa79753d6e4e49ad66cc32ba786fe8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.145874 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26aa79753d6e4e49ad66cc32ba786fe8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.146617 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26aa79753d6e4e49ad66cc32ba786fe8'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.325636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1371/2758] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:13:11 2022] DELETE /identity/v3/users/26aa79753d6e4e49ad66cc32ba786fe8 => generated 0 bytes in 550 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:13:12.654190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:12.667228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20edf7b4-c22c-4469-82c8-c7928b6c1ea6 service swift] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:12.767637 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:12.768632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:12.769096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:12.769553 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] PATH_INFO: `/v3/projects/ae31829bffd7483fbab3dc87635aea58` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:12.771188 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae31829bffd7483fbab3dc87635aea58)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:12.773369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:12.974842 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae31829bffd7483fbab3dc87635aea58', 'request_id': 'req-c54393e9-46e4-4284-b995-72c3e0ff409d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.983629 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae31829bffd7483fbab3dc87635aea58', 'request_id': 'req-c54393e9-46e4-4284-b995-72c3e0ff409d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:12.990290 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae31829bffd7483fbab3dc87635aea58', 'request_id': 'req-c54393e9-46e4-4284-b995-72c3e0ff409d'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.088816 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] The token cache is being invalidate because project ae31829bffd7483fbab3dc87635aea58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:13:13.089549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.092171 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.095392 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1388/2759] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:13:12 2022] DELETE /identity/v3/projects/ae31829bffd7483fbab3dc87635aea58 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:13:13.097208 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:13.106864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:13.157750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa5aae-570e-48ed-a2b3-77a59b32d1dd admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:13.267017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:13.267841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:13.268291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:13.268934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] PATH_INFO: `/v3/users/690c034eae2d4448a9b6de152728c0ca` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:13.270233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] RBAC: Authorizing `identity:delete_user(user_id=690c034eae2d4448a9b6de152728c0ca)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:13.272941 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:13.362757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:13.364863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:13.462551 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54393e9-46e4-4284-b995-72c3e0ff409d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:13.474262 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:13.474880 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:13.475149 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:13.475420 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:13.477307 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:13.490488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:13.493911 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:13.495528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:13.511121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1389/2760] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:13:13.603503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:13.635756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '690c034eae2d4448a9b6de152728c0ca'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.639347 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '690c034eae2d4448a9b6de152728c0ca'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.648813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '690c034eae2d4448a9b6de152728c0ca'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.667713 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '690c034eae2d4448a9b6de152728c0ca'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.668641 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '690c034eae2d4448a9b6de152728c0ca'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:13.780317 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1372/2761] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:13:13 2022] DELETE /identity/v3/users/690c034eae2d4448a9b6de152728c0ca => generated 0 bytes in 684 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:13:14.378450 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:14.380393 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:14.398682 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa60fc8-827b-4b6b-9074-fa66f401c325 service nova] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:14.459438 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:14.460060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:14.460365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:14.460634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] PATH_INFO: `/v3/projects/71512d420f5245c8976eadd51e306229` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:14.461732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71512d420f5245c8976eadd51e306229)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:14.464257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:14.677248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71512d420f5245c8976eadd51e306229', 'request_id': 'req-3e211938-7cf1-43ca-9326-eea565194370'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:14.684683 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71512d420f5245c8976eadd51e306229', 'request_id': 'req-3e211938-7cf1-43ca-9326-eea565194370'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:14.690813 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71512d420f5245c8976eadd51e306229', 'request_id': 'req-3e211938-7cf1-43ca-9326-eea565194370'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:14.818879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] The token cache is being invalidate because project 71512d420f5245c8976eadd51e306229 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:13:14.819167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:14.819806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-3e211938-7cf1-43ca-9326-eea565194370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:13:14.832747 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1390/2762] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:13:14 2022] DELETE /identity/v3/projects/71512d420f5245c8976eadd51e306229 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:13:18.852940 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:18.856609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:18.965719 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b8a3b-ac90-41d9-be90-29beac8a5b66 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:19.039416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:19.040296 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:19.040582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:19.040945 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:19.042927 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:19.103145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:19.118266 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:19.120885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:19.137462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1373/2763] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:20.627639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:20.628407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:20.628407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:20.653613 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] MFA Rules not processed for user `ae8540a6f24149abbab456ca1c6b33c7`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:13:20.654574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:20.725026 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1391/2764] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:13:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:21.482861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:21.484376 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:21.566004 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658ca96a-a585-4401-b87c-9fba6057154e service nova] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:21.571808 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:21.572549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:21.572842 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:21.573118 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:21.577423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:21.589414 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:21.591010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9f26f6-3fae-4f0c-91ad-ee049f165a19 service placement] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:21.602367 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1374/2765] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:29.445538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:29.445538 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:29.483279 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:29.508131 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a5c860-959b-40bd-aa71-016d1c698837 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:29.512501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:29.513104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:29.513381 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:29.513658 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:29.515380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:29.526015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:29.527448 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85076bc9-4646-4135-acef-17bfceb26c27 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:29.535877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1392/2766] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:30.065499 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:30.066038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:30.066148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] PATH_INFO: `/` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:30.067407 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1375/2767] 10.222.0.97 () {54 vars in 921 bytes} [Thu Sep 8 14:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 08 14:13:30.073929 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:30.074223 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:30.074502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:30.118237 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] MFA Rules not processed for user `46beefbd50f34b43b82977125f31ffeb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:13:30.119496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:30.130765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1393/2768] 10.222.0.97 () {58 vars in 1018 bytes} [Thu Sep 8 14:13:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:30.143577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:30.145350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:30.227844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84a0581-731c-4b37-ae15-08c5e80e60fb None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:30.232025 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:30.232650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:30.232935 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:30.233206 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:30.243378 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:30.245427 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:30.256416 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1376/2769] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:30.586853 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:30.597701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a56cef3-93cb-4d04-908a-81d8ef94d51d None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:30.602132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:30.608020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:30.608020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:30.608020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:30.608020 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:30.616022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:30.617493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f836b-8ff2-4219-9003-b3d68c6b8238 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:30.628454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1394/2770] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:13:30.717142 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:30.726782 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb22d31-7ac5-47b1-b269-414b17a47afe service glance] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:30.730371 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:30.731014 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:30.731295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:30.731568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:30.733333 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:30.747049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:30.748608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:30.759497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1377/2771] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:13:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:13:33.596772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13af8-4e39-45d8-a660-77766436a1e7 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:33.597308 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13af8-4e39-45d8-a660-77766436a1e7 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:33.597487 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e13af8-4e39-45d8-a660-77766436a1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:33.621589 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-94e13af8-4e39-45d8-a660-77766436a1e7 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:13:33.622821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-94e13af8-4e39-45d8-a660-77766436a1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:33.680346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1395/2772] 10.222.0.97 () {58 vars in 1009 bytes} [Thu Sep 8 14:13:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:33.706756 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:33.708357 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:33.756240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=100 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:33.783456 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a55f181-0a7d-427f-a7cb-4c74a269d032 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:33.788925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:33.789601 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:33.789903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:33.790182 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:33.799083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:33.801047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:33.809990 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1378/2773] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:13:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 08 14:13:43.673689 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=52 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:43.807049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=12 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:13:50.979543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:50.980206 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:50.980485 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:50.999601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] MFA Rules not processed for user `69ef2e0efb2143bab3ea9f3e12177331`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:13:51.000817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:13:51.007792 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1396/2774] 10.222.0.97 () {58 vars in 967 bytes} [Thu Sep 8 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:13:51.020835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:13:51.031534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2218efc6-a464-455c-a6ec-ed0cfe531565 service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:13:51.036695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '71620f9e7c3147829e384a5cfae55f07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:13:51.037295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:13:51.037602 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:13:51.037877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:13:51.047043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:13:51.048653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:13:51.057569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1379/2775] 10.222.0.97 () {60 vars in 1384 bytes} [Thu Sep 8 14:13:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 08 14:14:00.994196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:01.053529 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=6 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:36.379863 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:36.390330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79253a24-31f1-4f25-b2a6-d521c84d4e99 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:36.394071 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:36.394650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:36.395047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:36.395326 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:36.397183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:36.464109 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:36.465572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:36.475488 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1397/2776] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:40.321198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:40.322643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:40.338664 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc3903e-b7cc-43d3-a113-ded90ad6a5cc service swift] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:40.343387 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:40.343924 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:40.344227 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:40.344609 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] PATH_INFO: `/v3/users/f5e920d3b3044859b767add41690043c` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:40.345728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5e920d3b3044859b767add41690043c)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:40.347681 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:40.641902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5e920d3b3044859b767add41690043c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:40.642585 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5e920d3b3044859b767add41690043c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:40.650202 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5e920d3b3044859b767add41690043c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:40.662061 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5e920d3b3044859b767add41690043c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:40.662745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-8a23e20c-a482-4cb3-93a4-e2e25b881728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5e920d3b3044859b767add41690043c'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:40.782193 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1380/2777] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:40 2022] DELETE /identity/v3/users/f5e920d3b3044859b767add41690043c => generated 0 bytes in 462 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:41.058744 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:41.070111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948f3d86-7d8d-4119-bee1-3726392d11f5 service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:41.133467 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:41.134027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.134341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.134572 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] PATH_INFO: `/v3/projects/1abe0ab206094524af0fdde9e6285385` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.135671 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=1abe0ab206094524af0fdde9e6285385)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:41.137685 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:41.285802 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1abe0ab206094524af0fdde9e6285385', 'request_id': 'req-a6784b98-7e91-4af3-b193-e3b04791a5ab'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:41.293239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1abe0ab206094524af0fdde9e6285385', 'request_id': 'req-a6784b98-7e91-4af3-b193-e3b04791a5ab'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:41.299034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1abe0ab206094524af0fdde9e6285385', 'request_id': 'req-a6784b98-7e91-4af3-b193-e3b04791a5ab'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:41.398841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] The token cache is being invalidate because project 1abe0ab206094524af0fdde9e6285385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:41.399134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:41.399648 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a6784b98-7e91-4af3-b193-e3b04791a5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:41.401176 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1398/2778] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:41 2022] DELETE /identity/v3/projects/1abe0ab206094524af0fdde9e6285385 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:41.406452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.406937 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.407043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.431533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:41.431533 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:41.497769 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1381/2779] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:41.509777 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.510059 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.510331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.530765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:41.531957 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:41.540634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1399/2780] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:41.545322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:41.555276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec4ec04f-cbbf-4e00-88cf-eaa5ed96700c None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:41.559934 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:41.560574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.560857 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.561132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.561846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:14:41.562154 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:41.563796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:41.575961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1382/2781] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:14:41.580570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:41.590383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e761e9ae-e0aa-47f6-aae6-9bf01331e88a None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:41.595636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:41.596278 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.596556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.596821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.597579 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:41.599501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:41.619157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:41.721770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1400/2782] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:41 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:41.726223 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:41.736825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60cddef-6f0c-4f01-b955-50ae97e11195 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:41.794082 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:41.794687 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:41.795056 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:41.795242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:41.795870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:41.797942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:41.807093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:41.982405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1383/2783] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:41 2022] POST /identity/v3/users => generated 287 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:41.986836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:41.997199 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1f51c9-a2ad-46bf-a00c-ce837a2fe9d3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:42.001774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:42.002363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.002644 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.002917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.003651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:42.005898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:42.017878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1401/2784] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:42.021951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:42.032997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2859651-6861-44b9-8c1f-6144fa2517f7 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:42.037683 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:42.038283 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.038564 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.038836 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.039492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:42.041088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74711aa-a4f0-4653-bf65-59b9d6c92bf6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:42.051815 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1384/2785] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:42.061979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:42.073370 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac036a4-7508-4006-ac2e-6724d7bbb190 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:42.077897 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:42.078527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.078824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.079069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] PATH_INFO: `/v3/projects/2cc0861d0fab40d6913e832cccdab553/users/c4cc27d0544b49ceaf2e95aee1f1b841/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.094503 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2cc0861d0fab40d6913e832cccdab553, user_id=c4cc27d0544b49ceaf2e95aee1f1b841, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:42.097196 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:42.182157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1402/2786] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:42 2022] PUT /identity/v3/projects/2cc0861d0fab40d6913e832cccdab553/users/c4cc27d0544b49ceaf2e95aee1f1b841/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:42.193084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.193380 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.193797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.224411 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] MFA Rules not processed for user `c4cc27d0544b49ceaf2e95aee1f1b841`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:42.225958 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:42.296798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1385/2787] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:42.308598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:42.310451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:42.384956 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be236a3-a8d9-4013-b14c-4e52f3ee6c99 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:42.390454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '46beefbd50f34b43b82977125f31ffeb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'reader', 'member', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:42.391081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.391361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.391632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.400034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:42.401646 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac223c8-bdf7-43f2-8f84-6a13494bb8b5 service nova] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:42.410549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1403/2788] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:42.636280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:42.699806 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827bc19-79bd-46c1-aab5-b32ad1aa700f None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:42.704552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:42.705186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:42.705470 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:42.705787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] PATH_INFO: `/v3/users/c4cc27d0544b49ceaf2e95aee1f1b841` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:42.707040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4cc27d0544b49ceaf2e95aee1f1b841)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:42.708980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:43.001434 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4cc27d0544b49ceaf2e95aee1f1b841'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:43.002437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4cc27d0544b49ceaf2e95aee1f1b841'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:43.010270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4cc27d0544b49ceaf2e95aee1f1b841'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:43.020644 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:43.022980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4cc27d0544b49ceaf2e95aee1f1b841'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:43.023854 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4cc27d0544b49ceaf2e95aee1f1b841'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:43.125939 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1386/2789] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:42 2022] DELETE /identity/v3/users/c4cc27d0544b49ceaf2e95aee1f1b841 => generated 0 bytes in 490 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:43.130770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:43.131281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:43.131388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:43.151619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:43.152988 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:43.212972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1404/2790] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:43.222880 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:43.224437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:43.294073 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600a51a5-111b-41f6-a007-24464ffe1e59 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:43.298246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:43.298910 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:43.299293 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:43.299642 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:43.311540 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:43.313252 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693c3334-8984-4c23-9c4e-1b791c5af369 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:43.326748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1387/2791] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:43.897937 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:43.902096 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:43.915029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a01d44-2941-4d56-803d-73f98b316b05 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:43.922632 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:43.923518 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:43.924047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:43.925281 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] PATH_INFO: `/v3/projects/2cc0861d0fab40d6913e832cccdab553` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:43.926535 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2cc0861d0fab40d6913e832cccdab553)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:43.929357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.046762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:44.053030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2cc0861d0fab40d6913e832cccdab553', 'request_id': 'req-5ec488ba-beea-4475-8691-788ac52f09e5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.060127 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2cc0861d0fab40d6913e832cccdab553', 'request_id': 'req-5ec488ba-beea-4475-8691-788ac52f09e5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.066333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2cc0861d0fab40d6913e832cccdab553', 'request_id': 'req-5ec488ba-beea-4475-8691-788ac52f09e5'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.212512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] The token cache is being invalidate because project 2cc0861d0fab40d6913e832cccdab553 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:44.218464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.218464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-5ec488ba-beea-4475-8691-788ac52f09e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.218464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1405/2792] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:43 2022] DELETE /identity/v3/projects/2cc0861d0fab40d6913e832cccdab553 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:44.219984 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.220423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.220588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.241275 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:44.243215 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:44.302443 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1388/2793] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:44.311128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.311411 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.311690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.331599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:44.332844 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:44.342240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1406/2794] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:44.349492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.350033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.352734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:44.363476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceacb226-0932-457d-9efa-e7d7ab8ccada None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.367821 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a315abd4-730f-4146-b5cc-4dea0e7a0fb9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.371243 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.372039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.372245 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.372484 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.373140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:14:44.373457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.375231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.376008 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.376859 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.380766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.380766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] PATH_INFO: `/v3/users/ae8540a6f24149abbab456ca1c6b33c7` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.380766 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae8540a6f24149abbab456ca1c6b33c7)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.381190 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.391148 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1389/2795] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:14:44.395466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.406094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e973d-5057-418e-a127-6d4992754840 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.410870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.415235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.415235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.415235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.415235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.415235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.560285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1390/2796] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:44 2022] POST /identity/v3/projects => generated 403 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:44.564458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.596743 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae8540a6f24149abbab456ca1c6b33c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.597573 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae8540a6f24149abbab456ca1c6b33c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.605691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae8540a6f24149abbab456ca1c6b33c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.606483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae8540a6f24149abbab456ca1c6b33c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.624639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:44.626871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae8540a6f24149abbab456ca1c6b33c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:44.630016 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175667b-e78a-4ba0-bfa8-4555ea7e90a8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.634838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.635455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.635736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.636034 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.636745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.638697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.725086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1407/2797] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:44 2022] DELETE /identity/v3/users/ae8540a6f24149abbab456ca1c6b33c7 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:14:44.749738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1391/2798] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:44 2022] POST /identity/v3/users => generated 302 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:14:44.757677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.759121 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:44.828694 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ca03f5-ff20-4129-a020-5ddd8e22e88b admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.834039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.835106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.835603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.836100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.836836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.838495 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.850128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1408/2799] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 08 14:14:44.854791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.865048 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa9a868-f24d-477f-9337-acde39701de2 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.870794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.871392 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.871674 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.871942 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.872648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.874222 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:44.887957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1392/2800] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:44.889570 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:44.899857 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e7a07-507f-4fc8-9b9c-28697d8c6c8e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:44.904574 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:44.905226 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:44.905501 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:44.905772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] PATH_INFO: `/v3/projects/172c1978ad29467a844a5ad07c5c169d/users/1d08d9f270aa4a0a88e4ff78ce7fb220/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:44.921007 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=172c1978ad29467a844a5ad07c5c169d, user_id=1d08d9f270aa4a0a88e4ff78ce7fb220, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:44.923672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914a5900-b75c-451a-88c9-5666641654a2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.011528 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1409/2801] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:44 2022] PUT /identity/v3/projects/172c1978ad29467a844a5ad07c5c169d/users/1d08d9f270aa4a0a88e4ff78ce7fb220/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:45.016452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.018263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:45.024077 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.024650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.024936 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.033196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dece7f64-53fb-4ba6-a92d-9eebbcc295dd admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.039261 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:45.055621 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] MFA Rules not processed for user `1d08d9f270aa4a0a88e4ff78ce7fb220`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:45.056872 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:45.098542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.099176 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.099455 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.099730 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] PATH_INFO: `/v3/projects/d3cad599cd5943e8b545de7bacca6fe1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.100979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3cad599cd5943e8b545de7bacca6fe1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.103400 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.124603 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1410/2802] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:45.129316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.140419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fbc558-811f-426a-8b35-d2c68f666368 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.145122 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.145812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.146093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.146366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.147333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.150017 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.206224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3cad599cd5943e8b545de7bacca6fe1', 'request_id': 'req-15c81944-7424-4058-9be4-1b8aef6790c9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:45.213489 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3cad599cd5943e8b545de7bacca6fe1', 'request_id': 'req-15c81944-7424-4058-9be4-1b8aef6790c9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:45.219263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3cad599cd5943e8b545de7bacca6fe1', 'request_id': 'req-15c81944-7424-4058-9be4-1b8aef6790c9'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:45.281090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1411/2803] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:45 2022] POST /identity/v3/projects => generated 405 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:14:45.285100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.313397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] The token cache is being invalidate because project d3cad599cd5943e8b545de7bacca6fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88421) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:45.322669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:45.322669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:45.322669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1393/2804] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:45 2022] DELETE /identity/v3/projects/d3cad599cd5943e8b545de7bacca6fe1 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 08 14:14:45.357390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77af0b5-7ec4-41f9-9b7d-676ab7dfccdb admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.363248 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.364029 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.364354 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.364701 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] PATH_INFO: `/v3/users` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.365378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.367930 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.486624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1412/2805] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:45 2022] POST /identity/v3/users => generated 309 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:14:45.490546 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.492636 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:45.567312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c81944-7424-4058-9be4-1b8aef6790c9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.573315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.573948 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.574257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.574574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.575246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.576909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.582963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:45.588131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1394/2806] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:45.592925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.604773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95627c-652e-4d1e-ae9d-d6d7dabb2dbd admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.609374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.610015 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.610288 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.610564 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] PATH_INFO: `/v3/projects/45229c5ad6fb4ba3a85112b038d9d650/users/f19687964abb4412a6ac214f71692774/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.630628 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45229c5ad6fb4ba3a85112b038d9d650, user_id=f19687964abb4412a6ac214f71692774, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.633892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.730889 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1413/2807] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:45 2022] PUT /identity/v3/projects/45229c5ad6fb4ba3a85112b038d9d650/users/f19687964abb4412a6ac214f71692774/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:45.734967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.747009 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f084b-7e84-468c-a12c-13ea3c299cdb admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.805884 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.806495 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.806777 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.807137 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.807829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.809905 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffea06-6a8d-43ed-b4e0-d026ffc9ad98 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.820900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1395/2808] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:45 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:45.825210 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.835788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c4c184-7793-426b-8393-c936d9b1b503 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:45.841510 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:45.842156 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.842437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.842711 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] PATH_INFO: `/v3/projects/45229c5ad6fb4ba3a85112b038d9d650/users/f19687964abb4412a6ac214f71692774/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.844757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=45229c5ad6fb4ba3a85112b038d9d650, user_id=f19687964abb4412a6ac214f71692774, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:45.847387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:45.858805 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1414/2809] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:45 2022] PUT /identity/v3/projects/45229c5ad6fb4ba3a85112b038d9d650/users/f19687964abb4412a6ac214f71692774/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:45.869303 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:45.869588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:45.869888 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:45.900858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] MFA Rules not processed for user `f19687964abb4412a6ac214f71692774`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:45.902143 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:45.970925 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1396/2810] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:14:45.981605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:45.983400 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:46.015130 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:46.042426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b056fa19-1d86-46b6-aea8-780f6193a9ff admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:46.046274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:46.046877 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:46.047160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:46.047435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:46.058119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:46.059612 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84756ca4-ec12-4b45-93b8-f6a9d05acccf service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:46.070703 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1415/2811] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:46.224603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:46.287588 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c1bd37-1083-4b06-9610-2441d0874ac8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:46.292852 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:46.293481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:46.293765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:46.294039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] PATH_INFO: `/v3/users/1d08d9f270aa4a0a88e4ff78ce7fb220` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:46.295160 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d08d9f270aa4a0a88e4ff78ce7fb220)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:46.297127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:46.321100 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:46.590104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d08d9f270aa4a0a88e4ff78ce7fb220'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:46.590933 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d08d9f270aa4a0a88e4ff78ce7fb220'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:46.598165 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d08d9f270aa4a0a88e4ff78ce7fb220'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:46.610909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d08d9f270aa4a0a88e4ff78ce7fb220'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:46.611595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d08d9f270aa4a0a88e4ff78ce7fb220'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:46.724551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1397/2812] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:46 2022] DELETE /identity/v3/users/1d08d9f270aa4a0a88e4ff78ce7fb220 => generated 0 bytes in 500 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:46.729733 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:46.730252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:46.730252 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:46.750726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:46.751970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:46.810355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1416/2813] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:46.821170 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:46.822581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:46.886492 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3be2e-f225-4639-8f2d-8be5eb5decdc admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:46.890020 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:46.890628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:46.890936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:46.891236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:46.900285 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:46.901699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:46.911315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1398/2814] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:47.492755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:47.494198 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:47.512064 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0e665f-50ab-46c6-9901-04cfd90f9a5f None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:47.516770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:47.517453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:47.517770 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:47.518038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] PATH_INFO: `/v3/projects/172c1978ad29467a844a5ad07c5c169d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:47.519241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=172c1978ad29467a844a5ad07c5c169d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:47.521160 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:47.691619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '172c1978ad29467a844a5ad07c5c169d', 'request_id': 'req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:47.699183 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '172c1978ad29467a844a5ad07c5c169d', 'request_id': 'req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:47.705126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '172c1978ad29467a844a5ad07c5c169d', 'request_id': 'req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:47.775729 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] The token cache is being invalidate because project 172c1978ad29467a844a5ad07c5c169d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:47.776649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:47.776649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:47.778128 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1417/2815] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:47 2022] DELETE /identity/v3/projects/172c1978ad29467a844a5ad07c5c169d => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:47.782432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:47.792844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336fe8bd-20dc-4e51-a5e9-9f21a013be90 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:47.852270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:47.852899 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:47.854322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:47.854322 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] PATH_INFO: `/v3/users/f19687964abb4412a6ac214f71692774` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:47.854568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f19687964abb4412a6ac214f71692774)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:47.856551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:48.097811 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:48.099423 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f19687964abb4412a6ac214f71692774'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.100107 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f19687964abb4412a6ac214f71692774'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.107151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f19687964abb4412a6ac214f71692774'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.119420 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f19687964abb4412a6ac214f71692774'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.120103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-34607480-ec39-41a4-b214-ae8380748897 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f19687964abb4412a6ac214f71692774'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.205157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1399/2816] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:47 2022] DELETE /identity/v3/users/f19687964abb4412a6ac214f71692774 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:48.744167 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:48.745649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:48.760815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a65cac-df67-4fc0-bba6-f4eac1aa86c8 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:48.812124 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:48.821134 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:48.821769 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:48.827755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:48.827755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] PATH_INFO: `/v3/projects/45229c5ad6fb4ba3a85112b038d9d650` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:48.827755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45229c5ad6fb4ba3a85112b038d9d650)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:48.827755 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:48.983470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45229c5ad6fb4ba3a85112b038d9d650', 'request_id': 'req-cc2c2335-33c2-44d1-b94b-d6cf2a724984'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.990058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45229c5ad6fb4ba3a85112b038d9d650', 'request_id': 'req-cc2c2335-33c2-44d1-b94b-d6cf2a724984'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:48.995846 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45229c5ad6fb4ba3a85112b038d9d650', 'request_id': 'req-cc2c2335-33c2-44d1-b94b-d6cf2a724984'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:49.158952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] The token cache is being invalidate because project 45229c5ad6fb4ba3a85112b038d9d650 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:49.159292 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:49.159892 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-cc2c2335-33c2-44d1-b94b-d6cf2a724984 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:49.161406 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1418/2817] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:48 2022] DELETE /identity/v3/projects/45229c5ad6fb4ba3a85112b038d9d650 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:49.166595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.167277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.167277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.188595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:49.189861 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:49.254629 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1400/2818] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:49.264436 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.264728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.265006 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.285652 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:49.286860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:49.295132 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1419/2819] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:49.299929 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.310731 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02003791-fd87-43dc-8fe3-b5181fa47e56 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.317036 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.317667 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.317951 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.318228 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.318883 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:14:49.319187 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.321705 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.334146 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1401/2820] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:14:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 08 14:14:49.338540 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.349726 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f7ffd-e258-41f2-8677-f25310995d43 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.355115 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.355721 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.355998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.356331 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.356981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.358979 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.508175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1420/2821] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:49 2022] POST /identity/v3/projects => generated 389 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:49.509638 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.520076 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543f2843-0070-47df-989c-a22862b6760f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.578744 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.579589 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.579960 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.580365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.580998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.582963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.670572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1402/2822] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:49 2022] POST /identity/v3/users => generated 295 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:49.671789 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.683359 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec6abcd-cc3e-4bc5-bd77-c7dee268e915 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.688369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.688974 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.689287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.689561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.690212 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.691818 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.704508 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1421/2823] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:49.708830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.720099 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2695cd4a-705e-42c1-8fef-d50905732e6a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.725242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.725838 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.726188 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.727088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.727721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.729417 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e764cef-e795-41a9-9313-78b5167ba7d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.740171 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1403/2824] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:49.744809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:49.755531 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6953aa3-6f9b-460b-9ede-706e01d5121c admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:49.760316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:49.760906 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.761186 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.761455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] PATH_INFO: `/v3/projects/5465c194481f49a49863ba27939c98c8/users/9c68e0c40abe4d73941da0ff260cbf66/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.778433 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5465c194481f49a49863ba27939c98c8, user_id=9c68e0c40abe4d73941da0ff260cbf66, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:49.781102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:49.857651 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1422/2825] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:49 2022] PUT /identity/v3/projects/5465c194481f49a49863ba27939c98c8/users/9c68e0c40abe4d73941da0ff260cbf66/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:49.923396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:49.923682 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:49.923962 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:49.953235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] MFA Rules not processed for user `9c68e0c40abe4d73941da0ff260cbf66`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:49.954462 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:49.971619 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:50.019877 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1404/2826] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:50.025762 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.090240 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97708b-991a-4530-9d1d-251f03e27c63 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.095169 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.095767 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.096063 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.096352 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.096973 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.098855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.218502 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1423/2827] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:50 2022] POST /identity/v3/projects => generated 393 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:50.222699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.233586 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f19ad2d-4ec4-4876-9e6f-9dd8c688156b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.294568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.294568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.294568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.294568 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.295093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.295093 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.416365 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1405/2828] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:50 2022] POST /identity/v3/users => generated 303 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:50.420347 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.427090 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:50.432670 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062b1f1-b7ed-4242-a49e-1cc7479cb3a9 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.437668 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.438345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.438630 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.438908 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.439548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.441175 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.452870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1424/2829] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:50.458005 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.468831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95491c6-c000-4503-a96c-34618b4f3e61 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.473738 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.474386 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.474671 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.474949 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] PATH_INFO: `/v3/projects/101ae386fc6849bfbf1b487703d0a318/users/8f828a3e249b415f9e04e340567ae5cb/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.490131 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] RBAC: Authorizing `identity:create_grant(project_id=101ae386fc6849bfbf1b487703d0a318, user_id=8f828a3e249b415f9e04e340567ae5cb, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.493316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.582690 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1406/2830] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:50 2022] PUT /identity/v3/projects/101ae386fc6849bfbf1b487703d0a318/users/8f828a3e249b415f9e04e340567ae5cb/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:50.586723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.596866 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be17f148-b00f-4b2f-a9b9-e407ed2406a2 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.655241 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.655836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.656164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.656451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.657087 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.658662 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1428de-b01c-4ea1-a4d4-93634b3b6aba admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.668264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1425/2831] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:50.672628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.683487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d03e75e-1c2c-4ace-b47b-916e81afa20a admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.688159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.688840 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.689127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.689444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] PATH_INFO: `/v3/projects/101ae386fc6849bfbf1b487703d0a318/users/8f828a3e249b415f9e04e340567ae5cb/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.691758 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=101ae386fc6849bfbf1b487703d0a318, user_id=8f828a3e249b415f9e04e340567ae5cb, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.694697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.706277 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1407/2832] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:50 2022] PUT /identity/v3/projects/101ae386fc6849bfbf1b487703d0a318/users/8f828a3e249b415f9e04e340567ae5cb/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:50.716516 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.716834 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.717116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.745471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] MFA Rules not processed for user `8f828a3e249b415f9e04e340567ae5cb`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:50.746684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:50.815994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1426/2833] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:50.827487 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.829217 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:50.887251 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd85248-d862-4eb4-b653-933200ef0bd3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.891159 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.891794 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.892028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.892332 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:50.902263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:50.903737 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:50.913955 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1408/2834] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:14:50.961060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:50.963023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:50.975511 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4493eb-70b7-43f1-836f-64f705957eb2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:50.979422 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:50.980054 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:50.980328 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:50.980598 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:51.037998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:51.039423 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ca27f-66ec-4b8e-8b71-2997a909bd86 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:51.048608 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1427/2835] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:50 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:56.249732 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:56.263244 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:56.312661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f35056-632e-4378-9b4a-631751bb9c10 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:56.317310 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:56.317950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:56.318231 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:56.318506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] PATH_INFO: `/v3/users/9c68e0c40abe4d73941da0ff260cbf66` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:56.320259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c68e0c40abe4d73941da0ff260cbf66)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:56.321864 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:56.561659 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c68e0c40abe4d73941da0ff260cbf66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:56.562294 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c68e0c40abe4d73941da0ff260cbf66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:56.570506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c68e0c40abe4d73941da0ff260cbf66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:56.582574 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c68e0c40abe4d73941da0ff260cbf66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:56.583235 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c68e0c40abe4d73941da0ff260cbf66'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:56.678918 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1409/2836] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:56 2022] DELETE /identity/v3/users/9c68e0c40abe4d73941da0ff260cbf66 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:56.683676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:56.684251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:56.684251 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:56.703493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:56.707047 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:56.713428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:56.765723 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1428/2837] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:56.775691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:56.777151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:56.840872 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3790d814-a27b-4e46-8f49-9e7bb4e0aefa admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:56.844394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:56.845002 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:56.845280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:56.845556 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:56.855119 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:56.856550 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:56.866052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1410/2838] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:14:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:14:57.418027 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:57.419512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:57.433355 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821764c7-86fd-4e28-8b06-cf5e4339d2ef None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:57.439086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:57.439720 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:57.439999 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:57.440301 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] PATH_INFO: `/v3/projects/5465c194481f49a49863ba27939c98c8` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:57.441482 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5465c194481f49a49863ba27939c98c8)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:57.443365 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:57.571536 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5465c194481f49a49863ba27939c98c8', 'request_id': 'req-4bb09b00-c79b-40da-9d19-ffebdf564ad3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:57.578038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5465c194481f49a49863ba27939c98c8', 'request_id': 'req-4bb09b00-c79b-40da-9d19-ffebdf564ad3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:57.584003 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5465c194481f49a49863ba27939c98c8', 'request_id': 'req-4bb09b00-c79b-40da-9d19-ffebdf564ad3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:57.690490 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] The token cache is being invalidate because project 5465c194481f49a49863ba27939c98c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:57.691269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:57.691387 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:57.692817 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1429/2839] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:57 2022] DELETE /identity/v3/projects/5465c194481f49a49863ba27939c98c8 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:57.697321 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:57.707796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3d2c52-0239-48e4-94bf-7c9d1311040e service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:57.747161 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:57.773064 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:57.776444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:57.776444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:57.776444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] PATH_INFO: `/v3/users/8f828a3e249b415f9e04e340567ae5cb` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:57.776444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f828a3e249b415f9e04e340567ae5cb)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:57.777491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:58.023865 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f828a3e249b415f9e04e340567ae5cb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:58.024531 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f828a3e249b415f9e04e340567ae5cb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:58.031491 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f828a3e249b415f9e04e340567ae5cb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:58.043328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f828a3e249b415f9e04e340567ae5cb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:58.044295 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-31fea525-6250-4d6b-9721-0b20e6e0bf77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f828a3e249b415f9e04e340567ae5cb'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:58.170306 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1411/2840] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:14:57 2022] DELETE /identity/v3/users/8f828a3e249b415f9e04e340567ae5cb => generated 0 bytes in 473 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:58.800350 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:58.801918 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:58.816102 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb09b00-c79b-40da-9d19-ffebdf564ad3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:58.875900 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:58.876601 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:58.876887 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:58.877158 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] PATH_INFO: `/v3/projects/101ae386fc6849bfbf1b487703d0a318` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:58.878276 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=101ae386fc6849bfbf1b487703d0a318)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:58.880193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.028327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:59.035380 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '101ae386fc6849bfbf1b487703d0a318', 'request_id': 'req-f514627c-c41e-4efd-9482-df610072baf9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:59.042602 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '101ae386fc6849bfbf1b487703d0a318', 'request_id': 'req-f514627c-c41e-4efd-9482-df610072baf9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:59.048444 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '101ae386fc6849bfbf1b487703d0a318', 'request_id': 'req-f514627c-c41e-4efd-9482-df610072baf9'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:59.187104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] The token cache is being invalidate because project 101ae386fc6849bfbf1b487703d0a318 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:14:59.187333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:59.188000 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f514627c-c41e-4efd-9482-df610072baf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:14:59.189506 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1430/2841] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:14:58 2022] DELETE /identity/v3/projects/101ae386fc6849bfbf1b487703d0a318 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:59.194370 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.195648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.195869 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.216461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:59.217698 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:59.282512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1412/2842] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:59.292397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.292688 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.292971 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.313440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:59.314650 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:14:59.323005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1431/2843] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:14:59.327263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.336977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115ffcdf-78a2-4e63-beff-35a5a4ce7a72 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.341969 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.342595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.342886 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.343163 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.343810 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:14:59.344109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.345813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.358246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1413/2844] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:14:59.362539 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.373333 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639a877b-07f8-497c-bd0d-775748f93036 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.378104 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.378695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.378977 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.379253 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.379878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.381856 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.554860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1432/2845] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:14:59 2022] POST /identity/v3/projects => generated 373 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:59.559075 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.569394 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b78780-1831-4bba-8581-fbd9ffc01ba3 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.611482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:14:59.628363 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.628982 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.629273 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.629548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.630181 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.632157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.719121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1414/2846] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:14:59 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:14:59.723256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.734366 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97643466-a15f-4f94-9187-2b64c6cd4f7a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.738969 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.739550 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.739826 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.740093 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.740756 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.742319 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.752100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1433/2847] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:59.756246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.766466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461cdfb4-e09a-4589-911f-4142658fedea admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.771649 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.772286 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.772565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.773369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.773999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.775567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dc403f-8adc-4cfb-9a70-a6e578ab174c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.786527 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1415/2848] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:14:59.791140 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:14:59.801081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc59b9ea-0401-4b72-ae4e-fe8d13fa9538 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:14:59.806269 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:14:59.806878 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.807157 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.807427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] PATH_INFO: `/v3/projects/1f167909a2bb4fe3957b48914b1fc7c1/users/a870a8d1ba634b798c363142d10cc588/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.823148 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f167909a2bb4fe3957b48914b1fc7c1, user_id=a870a8d1ba634b798c363142d10cc588, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:14:59.825982 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:14:59.924458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1434/2849] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:14:59 2022] PUT /identity/v3/projects/1f167909a2bb4fe3957b48914b1fc7c1/users/a870a8d1ba634b798c363142d10cc588/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:14:59.929728 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabf1e-5925-4f85-baf5-8d4720f6f135 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:14:59.930012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabf1e-5925-4f85-baf5-8d4720f6f135 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:14:59.930291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6aabf1e-5925-4f85-baf5-8d4720f6f135 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:14:59.951096 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-d6aabf1e-5925-4f85-baf5-8d4720f6f135 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:14:59.952323 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-d6aabf1e-5925-4f85-baf5-8d4720f6f135 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:00.012593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1416/2850] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:15:00.022542 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:00.024919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:00.082919 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085a6e54-84c8-4cec-8d09-19a23259aea7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:00.087243 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:00.087833 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:00.088113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:00.088419 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:00.097593 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:00.099011 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:00.109249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1435/2851] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:01.674391 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:01.674987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:01.674987 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:01.708302 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] MFA Rules not processed for user `a870a8d1ba634b798c363142d10cc588`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:01.709469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:01.783044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1417/2852] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:01.788030 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:01.798491 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a94605-82b0-44d6-9835-3ceada90280f service neutron] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:01.803396 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:01.804524 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:01.804815 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:01.805229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:01.805915 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:01.808113 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:01.929374 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1436/2853] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:01 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:01.933432 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:01.945054 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3010800d-ce5a-4fb8-89b6-36c0e05687e7 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:01.994232 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:02.003800 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:02.004431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:02.004708 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:02.004977 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:02.005727 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:02.007645 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:02.165184 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1418/2854] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:01 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:02.169205 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:02.180004 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e11ab2-646b-48fe-be10-699d8ab0b85e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:02.184388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:02.184980 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:02.185264 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:02.185552 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:02.186254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:02.187845 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:02.199245 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1437/2855] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:02.207542 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:02.218569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d6e11-ce06-4d76-bb4c-7d61e9fb8234 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:02.223270 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:02.223916 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:02.224240 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:02.224514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] PATH_INFO: `/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c/users/c0d5e733d6d143aab3d8e14457f64acd/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:02.239436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d95b4408f1490fb1a8fac1a17e9e6c, user_id=c0d5e733d6d143aab3d8e14457f64acd, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:02.242126 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:02.375994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1419/2856] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:02 2022] PUT /identity/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c/users/c0d5e733d6d143aab3d8e14457f64acd/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:02.379931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:02.390238 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b02de05-f3ca-41c8-b333-47cae631058d admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:02.421563 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:02.449559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:02.450081 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:02.450363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:02.450964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:02.451636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:02.453287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45906e8-666c-429a-9cd4-75d232850fbf admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:02.463809 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1438/2857] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:02 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:02.468190 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:02.479530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a927a51-db7e-449f-84b8-9bac43622cd9 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:02.484807 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:02.496724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:02.496724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:02.496724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] PATH_INFO: `/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c/users/c0d5e733d6d143aab3d8e14457f64acd/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:02.496724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d95b4408f1490fb1a8fac1a17e9e6c, user_id=c0d5e733d6d143aab3d8e14457f64acd, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:02.496724 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:02.501532 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1420/2858] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:02 2022] PUT /identity/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c/users/c0d5e733d6d143aab3d8e14457f64acd/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:04.182384 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:04.182952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:04.182952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:04.212794 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] MFA Rules not processed for user `c0d5e733d6d143aab3d8e14457f64acd`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:04.214043 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:04.286477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1439/2859] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:04.299959 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:04.301784 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:04.363653 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef76e35-2cc5-441d-932e-ed4ce9b6557b admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:04.367268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:04.367873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:04.368194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:04.368475 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:04.427606 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:04.429121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:04.438140 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1421/2860] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:06.392263 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:06.393649 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:06.406471 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3185f0-d3fc-43ce-8ad7-57e140e5b459 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:06.410098 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:06.410690 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:06.410964 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:06.411231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:06.420231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:06.421636 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:06.431369 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1440/2861] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:06 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:14.434210 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=69,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:16.427012 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=54 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:24.155044 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:24.222452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757ff56-bb02-4666-8824-56de34e961d7 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:24.227736 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:24.228397 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:24.228678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:24.228953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] PATH_INFO: `/v3/users/a870a8d1ba634b798c363142d10cc588` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:24.230083 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a870a8d1ba634b798c363142d10cc588)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:24.232094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:24.614408 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a870a8d1ba634b798c363142d10cc588'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:24.614961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a870a8d1ba634b798c363142d10cc588'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:24.624039 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a870a8d1ba634b798c363142d10cc588'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:24.637144 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a870a8d1ba634b798c363142d10cc588'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:24.637870 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a870a8d1ba634b798c363142d10cc588'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:24.804239 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1422/2862] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:24 2022] DELETE /identity/v3/users/a870a8d1ba634b798c363142d10cc588 => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:25.096895 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:25.107970 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb891be-582f-4dff-8594-562457720755 service cinder] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:25.168363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:25.168921 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:25.169204 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:25.169481 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] PATH_INFO: `/v3/projects/1f167909a2bb4fe3957b48914b1fc7c1` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:25.170735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f167909a2bb4fe3957b48914b1fc7c1)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:25.172645 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:25.336673 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f167909a2bb4fe3957b48914b1fc7c1', 'request_id': 'req-f270331b-2913-4a9c-8ccb-d3f6823464c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.343213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f167909a2bb4fe3957b48914b1fc7c1', 'request_id': 'req-f270331b-2913-4a9c-8ccb-d3f6823464c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.349619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f167909a2bb4fe3957b48914b1fc7c1', 'request_id': 'req-f270331b-2913-4a9c-8ccb-d3f6823464c7'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.445227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] The token cache is being invalidate because project 1f167909a2bb4fe3957b48914b1fc7c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:25.445227 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.446287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.448117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1441/2863] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:25 2022] DELETE /identity/v3/projects/1f167909a2bb4fe3957b48914b1fc7c1 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:25.452282 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:25.461316 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:25.468701 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caceb7dc-b556-430d-8942-376d940b9ba0 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:25.526712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:25.527366 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:25.527691 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:25.527988 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] PATH_INFO: `/v3/users/c0d5e733d6d143aab3d8e14457f64acd` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:25.529191 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0d5e733d6d143aab3d8e14457f64acd)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:25.531105 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:25.553980 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:25.800860 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0d5e733d6d143aab3d8e14457f64acd'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.801535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0d5e733d6d143aab3d8e14457f64acd'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.808971 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0d5e733d6d143aab3d8e14457f64acd'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.822328 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0d5e733d6d143aab3d8e14457f64acd'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.823012 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-7b1020d7-f6b5-46e5-87ee-e57f2490c93a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0d5e733d6d143aab3d8e14457f64acd'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:25.906745 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1423/2864] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:25 2022] DELETE /identity/v3/users/c0d5e733d6d143aab3d8e14457f64acd => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:26.217765 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:26.219363 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:26.237634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f270331b-2913-4a9c-8ccb-d3f6823464c7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:26.295749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:26.296407 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:26.296691 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:26.296972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] PATH_INFO: `/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:26.298126 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d95b4408f1490fb1a8fac1a17e9e6c)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:26.300201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:26.519151 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:26.531316 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d95b4408f1490fb1a8fac1a17e9e6c', 'request_id': 'req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:26.540123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d95b4408f1490fb1a8fac1a17e9e6c', 'request_id': 'req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:26.546341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d95b4408f1490fb1a8fac1a17e9e6c', 'request_id': 'req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:26.700814 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] The token cache is being invalidate because project b7d95b4408f1490fb1a8fac1a17e9e6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:26.701106 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:26.701803 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-2001bcb4-fd84-4bce-a9d2-c4c2e3cd9854 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:26.703299 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1442/2865] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:26 2022] DELETE /identity/v3/projects/b7d95b4408f1490fb1a8fac1a17e9e6c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:26.708377 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:26.708900 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:26.709021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:26.730196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:26.731502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:26.795742 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1424/2866] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:26.804197 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:26.804591 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:26.804867 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:26.826085 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:26.827298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:26.835836 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1443/2867] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:26.840353 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:26.850111 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4546baa-d499-4730-a57f-9a3155d567e8 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:26.854748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:26.855339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:26.855614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:26.855889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:26.856565 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:15:26.856868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:26.858436 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:26.870535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1425/2868] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:15:26.874886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:26.885222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a6b98-d7ca-4436-b039-c3640205f008 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:26.889925 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:26.890519 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:26.890829 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:26.891105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:26.891731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:26.893672 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.017960 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1444/2869] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:26 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:27.023234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.033703 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baa523f-ef08-4987-8db4-854e873996d6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.093643 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.094136 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.094419 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.095017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.095689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.097628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.205579 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1426/2870] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:27 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:27.209879 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.221103 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6531d710-7f24-4835-a649-e371dbb7f554 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.226371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.226972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.227254 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.227527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.228193 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.229773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.239748 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1445/2871] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:27.244593 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.254661 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a5e6-a3ec-4c11-96c6-4aa79665dab4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.259229 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.259817 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.260094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.260396 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.261028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.262868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eee7cf-0f2a-43d4-ae60-b5ff03d941dc admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.273437 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1427/2872] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:27.277543 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.288222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eb3183-e3f1-4eeb-ab92-80d70798545a admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.292946 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.293716 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.294005 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.294280 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] PATH_INFO: `/v3/projects/b68af731780d42799fbf134fd5a95093/users/db4562391c18479193f91a48567700e7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.309514 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68af731780d42799fbf134fd5a95093, user_id=db4562391c18479193f91a48567700e7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.312211 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.404468 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1446/2873] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:27 2022] PUT /identity/v3/projects/b68af731780d42799fbf134fd5a95093/users/db4562391c18479193f91a48567700e7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:27.415502 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.415787 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.416060 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.426512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:27.445678 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] MFA Rules not processed for user `db4562391c18479193f91a48567700e7`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:27.446476 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:27.511289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1428/2874] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:27.515806 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.578117 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6df7d7-01c0-40f1-8d9b-8f1b4f503cea admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.582868 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.583458 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.583740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.584019 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.585213 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.587101 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.737075 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1447/2875] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:27 2022] POST /identity/v3/projects => generated 383 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:27.741315 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.752793 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071a1722-9c5a-43fb-b311-774d9a138bff None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.812038 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.812672 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.812953 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.813225 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.813919 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.815909 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.930507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1429/2876] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:27 2022] POST /identity/v3/users => generated 298 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:27.934718 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.945643 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbf24-f7fb-4000-8ab7-4af30561efaa admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.950275 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.950871 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.951154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.951426 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:27.952100 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:27.953725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:27.965695 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1448/2877] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:27.970608 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:27.990221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f6655b-1805-4a7f-acd5-0c902527f139 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:27.996124 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:27.996873 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:27.997153 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:27.997430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] PATH_INFO: `/v3/projects/1392a7467a1c4e89b01a814978d072d3/users/bfa7fc067f2849b1b1bdf2e55958e3fe/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:28.015088 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392a7467a1c4e89b01a814978d072d3, user_id=bfa7fc067f2849b1b1bdf2e55958e3fe, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:28.017828 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:28.148052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1430/2878] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:27 2022] PUT /identity/v3/projects/1392a7467a1c4e89b01a814978d072d3/users/bfa7fc067f2849b1b1bdf2e55958e3fe/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:28.152286 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:28.162416 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb329ba6-a79a-496f-9843-583203681a96 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:28.192267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:28.221341 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:28.222041 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:28.222220 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:28.222483 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:28.223112 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:28.224731 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0868b6b9-eb38-4fdf-9605-0fdd07ec7b1a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:28.234972 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1449/2879] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:28.239147 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:28.250297 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0b0e1f-ec37-46e7-a4d8-85c27964a7ff admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:28.255514 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:28.256289 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:28.257145 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:28.257440 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] PATH_INFO: `/v3/projects/1392a7467a1c4e89b01a814978d072d3/users/bfa7fc067f2849b1b1bdf2e55958e3fe/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:28.259422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1392a7467a1c4e89b01a814978d072d3, user_id=bfa7fc067f2849b1b1bdf2e55958e3fe, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:28.262047 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:28.273112 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1431/2880] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:28 2022] PUT /identity/v3/projects/1392a7467a1c4e89b01a814978d072d3/users/bfa7fc067f2849b1b1bdf2e55958e3fe/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:28.284610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:28.284894 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:28.285166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:28.314287 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] MFA Rules not processed for user `bfa7fc067f2849b1b1bdf2e55958e3fe`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:28.315504 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:28.387773 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1450/2881] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:28.400110 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:28.401469 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:28.459647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:28.465221 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea9ae1-bf74-46a4-bdf6-5ecd8639a9b6 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:28.469057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:28.469697 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:28.469981 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:28.470257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:28.532917 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:28.534473 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:28.543979 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1432/2882] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:15:34.350704 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:34.352177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:34.364345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec31326-a5b9-4a1f-8618-4e0452007d4e None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:34.368532 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:34.369119 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:34.369395 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:34.369661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:34.379246 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:34.380788 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72ed00-95de-4ee5-92ea-1df95fc514ad service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:34.391256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1451/2883] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 08 14:15:38.746530 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=27 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:40.823512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:40.891194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eca83cf-51fc-4148-90b5-62a75ede54b0 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:40.897650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:40.898339 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:40.898614 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:40.898892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] PATH_INFO: `/v3/users/db4562391c18479193f91a48567700e7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:40.900085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db4562391c18479193f91a48567700e7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:40.902030 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:41.117616 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db4562391c18479193f91a48567700e7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:41.118214 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db4562391c18479193f91a48567700e7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:41.125688 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db4562391c18479193f91a48567700e7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:41.139028 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db4562391c18479193f91a48567700e7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:41.139763 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db4562391c18479193f91a48567700e7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:41.238903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1433/2884] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:40 2022] DELETE /identity/v3/users/db4562391c18479193f91a48567700e7 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:41.243784 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:41.244378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:41.244378 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:41.264397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:41.265775 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:41.333177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1452/2885] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:41.344101 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:41.345603 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:41.407057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbec278-a9f0-4aea-ab05-766494dfc1d4 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:41.411549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:41.412517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:41.413430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:41.413748 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:41.424595 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:41.426233 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:41.437465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1434/2886] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:41.921935 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:41.923749 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:41.938139 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1361c1cd-e94e-4002-ac59-ae438baf39f3 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:41.943330 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:41.944024 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:41.944343 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:41.944619 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] PATH_INFO: `/v3/projects/b68af731780d42799fbf134fd5a95093` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:41.946168 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68af731780d42799fbf134fd5a95093)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:41.948202 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:42.081625 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:42.092256 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68af731780d42799fbf134fd5a95093', 'request_id': 'req-e592025a-365a-41bf-b04b-7b00cddab55f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.099036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68af731780d42799fbf134fd5a95093', 'request_id': 'req-e592025a-365a-41bf-b04b-7b00cddab55f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.105270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68af731780d42799fbf134fd5a95093', 'request_id': 'req-e592025a-365a-41bf-b04b-7b00cddab55f'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.231952 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] The token cache is being invalidate because project b68af731780d42799fbf134fd5a95093 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:42.232725 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.233843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.236785 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1453/2887] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:41 2022] DELETE /identity/v3/projects/b68af731780d42799fbf134fd5a95093 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:42.242442 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:42.252483 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e603c9e-2fea-4649-906b-8a2cc8c93095 service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:42.275517 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:42.321404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:42.322961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:42.322961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:42.322961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] PATH_INFO: `/v3/users/bfa7fc067f2849b1b1bdf2e55958e3fe` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:42.324791 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bfa7fc067f2849b1b1bdf2e55958e3fe)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:42.327465 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:42.659572 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bfa7fc067f2849b1b1bdf2e55958e3fe'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.660348 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bfa7fc067f2849b1b1bdf2e55958e3fe'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.669052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bfa7fc067f2849b1b1bdf2e55958e3fe'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.682751 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bfa7fc067f2849b1b1bdf2e55958e3fe'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.683482 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-1941d3b1-4ca8-49ed-a62c-930e6d7894d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bfa7fc067f2849b1b1bdf2e55958e3fe'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:42.816220 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1435/2888] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:42 2022] DELETE /identity/v3/users/bfa7fc067f2849b1b1bdf2e55958e3fe => generated 0 bytes in 574 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:43.285377 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:43.287060 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:43.301257 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e592025a-365a-41bf-b04b-7b00cddab55f admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:43.367155 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:43.367791 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:43.368069 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:43.368371 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] PATH_INFO: `/v3/projects/1392a7467a1c4e89b01a814978d072d3` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:43.369529 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] RBAC: Authorizing `identity:delete_project(project_id=1392a7467a1c4e89b01a814978d072d3)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:43.371428 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:43.519023 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1392a7467a1c4e89b01a814978d072d3', 'request_id': 'req-6d5f972e-ced0-485b-8ae9-f1318b86cebc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:43.526639 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1392a7467a1c4e89b01a814978d072d3', 'request_id': 'req-6d5f972e-ced0-485b-8ae9-f1318b86cebc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:43.532565 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1392a7467a1c4e89b01a814978d072d3', 'request_id': 'req-6d5f972e-ced0-485b-8ae9-f1318b86cebc'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:43.666559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] The token cache is being invalidate because project 1392a7467a1c4e89b01a814978d072d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:43.666855 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:43.667527 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-6d5f972e-ced0-485b-8ae9-f1318b86cebc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:43.669072 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1454/2889] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:43 2022] DELETE /identity/v3/projects/1392a7467a1c4e89b01a814978d072d3 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:43.675010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:43.675480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:43.675582 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:43.697010 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:43.698234 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:43.759109 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1436/2890] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 08 14:15:43.767496 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:43.767812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:43.768195 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:43.789998 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:43.791250 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:43.800229 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1455/2891] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:43.805510 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:43.816309 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1875fa2b-162a-4854-a3d6-f9e488fbdfaa None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:43.821479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:43.822089 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:43.822369 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:43.822639 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:43.823276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:15:43.823577 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:43.825242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:43.838892 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1437/2892] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:15:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:15:43.843870 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:43.854239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a36a7ec-e33d-49b7-aa3f-6a87bef6f5b9 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:43.859052 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:43.859677 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:43.859959 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:43.860267 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:43.860931 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:43.862827 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:43.988604 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1456/2893] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:43 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:43.993647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.004006 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300972-5279-48fd-8058-8d0745e89082 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.038098 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:44.071335 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.071858 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.072156 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.072458 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.073091 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.075043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.175646 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1438/2894] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:43 2022] POST /identity/v3/users => generated 292 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:44.179942 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.190609 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4052ef1f-e5fa-4fde-a9cc-d18cf1b924a5 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.195447 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.196044 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.196440 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.196684 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.197321 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.198920 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.209843 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1457/2895] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:44.213936 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.224172 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb96b295-0150-4f89-858c-6f5c7f024934 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.229584 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.230194 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.230479 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.230785 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.231430 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.233050 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858900fb-c4b3-41fb-a429-6c29c885b8fb admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.243626 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1439/2896] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:44.247993 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.258274 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebbc44b-05eb-466e-92b3-b97b5dadd422 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.263201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.263891 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.264192 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.264470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] PATH_INFO: `/v3/projects/c0ced1692d724a078d2acd97841549b2/users/6f6d459d662a45d59a6310570dc325a7/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.281194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ced1692d724a078d2acd97841549b2, user_id=6f6d459d662a45d59a6310570dc325a7, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.281719 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:44.284873 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.392742 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1458/2897] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:44 2022] PUT /identity/v3/projects/c0ced1692d724a078d2acd97841549b2/users/6f6d459d662a45d59a6310570dc325a7/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:44.405902 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.406189 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.406459 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.438199 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] MFA Rules not processed for user `6f6d459d662a45d59a6310570dc325a7`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:44.439422 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:44.511573 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1440/2898] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:44.516338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.586807 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9751ec-98c7-478c-806c-9954825e1bc7 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.592432 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.593048 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.593327 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.593599 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.594231 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.596133 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.700486 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1459/2899] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:44 2022] POST /identity/v3/projects => generated 383 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:44.705268 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.721549 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b4aabb-1b6d-48d3-b66f-d5eabd933fe2 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.792648 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.793256 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.793581 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.793859 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.794496 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.796538 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.947788 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1441/2900] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:44 2022] POST /identity/v3/users => generated 292 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:44.951882 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.962464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df19baa8-7205-4895-b1d3-8c9e9854f297 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:44.967166 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:44.967757 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:44.968038 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:44.968346 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:44.969058 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:44.970556 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:44.982752 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1460/2901] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:44.988151 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:44.999071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf1e745-731f-4cf1-bd92-ac6268f688da admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.003841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.004481 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.004764 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.005043 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.005685 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.007961 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2803361-5fc1-433b-84c7-c7d722b41759 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.018224 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1442/2902] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:45.022635 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.032512 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fd820a-48e5-4681-aecf-5a651421ad01 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.037036 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.037676 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.037948 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.038216 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] PATH_INFO: `/v3/projects/fc41dc57bb9e407dad625b69b2d2fd02/users/c1138d7d9c224b028b50c2120c14d0a1/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.053561 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc41dc57bb9e407dad625b69b2d2fd02, user_id=c1138d7d9c224b028b50c2120c14d0a1, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.056451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.141296 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1461/2903] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:45 2022] PUT /identity/v3/projects/fc41dc57bb9e407dad625b69b2d2fd02/users/c1138d7d9c224b028b50c2120c14d0a1/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:45.151752 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.152040 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.152346 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.182421 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] MFA Rules not processed for user `c1138d7d9c224b028b50c2120c14d0a1`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:45.183647 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:45.187796 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:45.252755 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1443/2904] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:45 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:45.257339 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.320917 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e03b6f0-85c4-4411-833a-4d74ef436495 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.325530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.326123 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.326398 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.326666 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.327322 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.329260 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.445876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1462/2905] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:45 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:45.450198 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.461457 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d2a47-6bee-493f-8b04-c2b6c2fd048b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.519721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.520284 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.520569 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.520846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.521567 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.523578 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.639049 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1444/2906] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:45 2022] POST /identity/v3/users => generated 298 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:45.643357 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.654283 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b4992-2233-43ad-b617-094ecd74e0c0 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.658886 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.659523 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.659811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.660084 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.660745 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.662989 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.673437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1463/2907] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:45.680972 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.692650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e4278b-2909-424f-bf8a-d27d0d198f60 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.698132 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.699084 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.699410 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.699829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] PATH_INFO: `/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea/users/7eac18d679d14dccae9eae2e1fb69f7b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.717835 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993f37a2ade94cf297b6c3c2f975f8ea, user_id=7eac18d679d14dccae9eae2e1fb69f7b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.720640 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.849776 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1445/2908] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:45 2022] PUT /identity/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea/users/7eac18d679d14dccae9eae2e1fb69f7b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:45.853981 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.864222 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9c1043-86e9-41af-af01-4f60b7840389 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.874228 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:45.926194 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.927230 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.927734 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.928754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.929884 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.932812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1c3613-e4c1-4d1f-acb3-f61e9f94c5a1 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.952820 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1464/2909] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:45.957094 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:45.967813 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7330121b-2582-4b5b-8d81-1141f524eaa8 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:45.972507 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:45.973157 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] REQUEST_METHOD: `PUT` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:45.973439 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:45.973789 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] PATH_INFO: `/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea/users/7eac18d679d14dccae9eae2e1fb69f7b/roles/e7c0471ae3484d85b6e31650411b03e6` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:45.976246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993f37a2ade94cf297b6c3c2f975f8ea, user_id=7eac18d679d14dccae9eae2e1fb69f7b, role_id=e7c0471ae3484d85b6e31650411b03e6)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:45.978846 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:45.989890 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1446/2910] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:45 2022] PUT /identity/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea/users/7eac18d679d14dccae9eae2e1fb69f7b/roles/e7c0471ae3484d85b6e31650411b03e6 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:46.000771 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:46.000955 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:46.001185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:46.030185 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] MFA Rules not processed for user `7eac18d679d14dccae9eae2e1fb69f7b`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:46.033361 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:46.105138 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1465/2911] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:45 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:46.117722 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:46.119405 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:46.176985 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce41bbe-b01c-4b03-a009-fa4f4cd7e453 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:46.180721 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:46.181395 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:46.181607 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:46.181897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:46.192092 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:46.241125 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:46.242576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:46.251669 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1447/2912] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:46.700358 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:46.701759 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:46.772272 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e683541e-01d8-4493-aaff-dd348e18e9cc None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:46.777435 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:46.778033 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:46.778349 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:46.778624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:46.780464 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:46.792810 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:46.794390 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:46.806472 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1466/2913] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:48.311611 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:48.313138 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:48.325057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ccb75a-6f68-49c3-b49a-613717cd9a2d service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:48.328699 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:48.331725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:48.331725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:48.331725 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:48.389307 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:48.390798 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:48.400868 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1448/2914] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:48 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:50.195804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:50.205493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef00a09-6102-4fb5-87f6-82ac7913ad5f service placement] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:50.209654 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eb8f5ef6eb4948edbf947eacffea57bb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:50.210249 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:50.210530 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:50.210841 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:50.220990 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:50.222455 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4671094-424f-49fa-ade6-e23b19a474c2 service cinder] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:50.233105 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1467/2915] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:55.159628 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:55.223127 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4110a81-0dc9-444e-972f-243beab31747 service cinder] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:55.227832 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:55.228477 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:55.228754 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:55.229021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] PATH_INFO: `/v3/users/6f6d459d662a45d59a6310570dc325a7` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:55.230123 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f6d459d662a45d59a6310570dc325a7)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:55.231994 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:55.441052 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f6d459d662a45d59a6310570dc325a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:55.441830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f6d459d662a45d59a6310570dc325a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:55.448519 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:55.451398 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f6d459d662a45d59a6310570dc325a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:55.465242 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f6d459d662a45d59a6310570dc325a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:55.465954 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f6d459d662a45d59a6310570dc325a7'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:55.608830 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1449/2916] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:55 2022] DELETE /identity/v3/users/6f6d459d662a45d59a6310570dc325a7 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:55.613478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:55.614016 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:55.614120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:55.634298 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:55.635499 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:55.640740 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=100 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:55.694522 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1468/2917] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:55.704558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:55.705997 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:55.764103 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416917c2-cf54-400e-aeb8-c07acf1c8c1c admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:55.767625 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:55.768259 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:55.768535 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:55.768803 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:55.777750 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:55.779162 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:55.789037 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1450/2918] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:15:56.257797 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:56.259427 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:56.274865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47486f57-78bd-459c-9292-478f7ad9a8ea None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:56.279610 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:56.283459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:56.283459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:56.283459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] PATH_INFO: `/v3/projects/c0ced1692d724a078d2acd97841549b2` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:56.283459 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ced1692d724a078d2acd97841549b2)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:56.284397 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:56.417270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ced1692d724a078d2acd97841549b2', 'request_id': 'req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.426617 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ced1692d724a078d2acd97841549b2', 'request_id': 'req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.431161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ced1692d724a078d2acd97841549b2', 'request_id': 'req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.529300 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] The token cache is being invalidate because project c0ced1692d724a078d2acd97841549b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:56.529578 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.530201 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.531735 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1469/2919] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:56 2022] DELETE /identity/v3/projects/c0ced1692d724a078d2acd97841549b2 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:56.535862 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:56.546998 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9991e45-2466-4b3c-9b21-8f659e4d4c0b service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:56.612246 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:56.612897 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:56.613178 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:56.613452 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] PATH_INFO: `/v3/users/c1138d7d9c224b028b50c2120c14d0a1` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:56.614762 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1138d7d9c224b028b50c2120c14d0a1)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:56.617478 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:56.812957 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1138d7d9c224b028b50c2120c14d0a1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.813712 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1138d7d9c224b028b50c2120c14d0a1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.821726 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1138d7d9c224b028b50c2120c14d0a1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.835466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1138d7d9c224b028b50c2120c14d0a1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.836438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1138d7d9c224b028b50c2120c14d0a1'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:56.838263 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:56.928978 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1451/2920] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:56 2022] DELETE /identity/v3/users/c1138d7d9c224b028b50c2120c14d0a1 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:57.450994 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:57.452559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:57.466709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63789b2e-8ecd-465b-b2f6-9c51721d7dc3 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:57.527507 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:57.528129 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:57.528429 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:57.528697 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] PATH_INFO: `/v3/projects/fc41dc57bb9e407dad625b69b2d2fd02` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:57.530161 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc41dc57bb9e407dad625b69b2d2fd02)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:57.532034 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:57.676837 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:57.687203 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc41dc57bb9e407dad625b69b2d2fd02', 'request_id': 'req-a375c68a-fd0c-4b94-8bc4-94771b867db6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:57.696470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc41dc57bb9e407dad625b69b2d2fd02', 'request_id': 'req-a375c68a-fd0c-4b94-8bc4-94771b867db6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:57.704454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc41dc57bb9e407dad625b69b2d2fd02', 'request_id': 'req-a375c68a-fd0c-4b94-8bc4-94771b867db6'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:57.803548 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] The token cache is being invalidate because project fc41dc57bb9e407dad625b69b2d2fd02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:57.803822 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:57.804477 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:57.810055 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1470/2921] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:57 2022] DELETE /identity/v3/projects/fc41dc57bb9e407dad625b69b2d2fd02 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:57.810324 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:57.811428 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:57.824963 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6d8c64-6a6c-45d3-95a3-4aae005d9e0f admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:57.883695 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:57.884424 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:57.884702 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:57.885117 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] PATH_INFO: `/v3/users/7eac18d679d14dccae9eae2e1fb69f7b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:57.886291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eac18d679d14dccae9eae2e1fb69f7b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:57.888262 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:58.134792 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eac18d679d14dccae9eae2e1fb69f7b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:58.135444 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eac18d679d14dccae9eae2e1fb69f7b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:58.142797 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eac18d679d14dccae9eae2e1fb69f7b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:58.154841 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eac18d679d14dccae9eae2e1fb69f7b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:58.155560 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-3e5d867c-d52f-45d1-934a-64fdbecfab18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eac18d679d14dccae9eae2e1fb69f7b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:58.238534 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1452/2922] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:15:57 2022] DELETE /identity/v3/users/7eac18d679d14dccae9eae2e1fb69f7b => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:58.888270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:58.888270 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:58.901916 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375c68a-fd0c-4b94-8bc4-94771b867db6 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:58.971661 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:58.973750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:58.973750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:58.973750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] PATH_INFO: `/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:58.974086 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993f37a2ade94cf297b6c3c2f975f8ea)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:58.976010 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.262111 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993f37a2ade94cf297b6c3c2f975f8ea', 'request_id': 'req-7c4e7c0f-1332-41b8-823b-03fd66d55e04'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:59.275659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993f37a2ade94cf297b6c3c2f975f8ea', 'request_id': 'req-7c4e7c0f-1332-41b8-823b-03fd66d55e04'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:59.282663 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993f37a2ade94cf297b6c3c2f975f8ea', 'request_id': 'req-7c4e7c0f-1332-41b8-823b-03fd66d55e04'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:59.414520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] The token cache is being invalidate because project 993f37a2ade94cf297b6c3c2f975f8ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:15:59.414811 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:59.415454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7c4e7c0f-1332-41b8-823b-03fd66d55e04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:15:59.417120 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1471/2923] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:15:58 2022] DELETE /identity/v3/projects/993f37a2ade94cf297b6c3c2f975f8ea => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:15:59.422715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.423280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.423280 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.444404 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:59.445878 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:59.506765 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1453/2924] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:59.515898 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.516116 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.516453 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.537513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:15:59.538876 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:15:59.546753 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1472/2925] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:15:59.551017 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.561431 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59864236-f810-4b7f-9449-734b7b67c09b None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.565950 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.566715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.566991 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.567261 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] PATH_INFO: `/v3/domains` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.567904 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=88421) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 08 14:15:59.568312 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.569889 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.576999 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:59.581967 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1454/2926] 10.222.0.97 () {58 vars in 1153 bytes} [Thu Sep 8 14:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 08 14:15:59.586388 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.595772 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d35459c-dcc5-4c15-b6a2-c720627464d2 None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.600932 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.601520 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.601865 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.602118 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] PATH_INFO: `/v3/projects` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.602750 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.604659 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.702754 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1473/2927] 10.222.0.97 () {60 vars in 1153 bytes} [Thu Sep 8 14:15:59 2022] POST /identity/v3/projects => generated 369 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 08 14:15:59.707425 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.717598 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9329c13e-97b5-44a0-b608-e6b5d606f277 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.776689 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.777236 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.777552 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.777825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] PATH_INFO: `/v3/users` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.778466 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.780480 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.878254 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1455/2928] 10.222.0.97 () {60 vars in 1144 bytes} [Thu Sep 8 14:15:59 2022] POST /identity/v3/users => generated 285 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 08 14:15:59.888446 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.899732 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0f192f-32fe-46d4-a516-98ceee71d33e admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.904566 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.905154 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.905437 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.905736 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] PATH_INFO: `/v3/roles` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.906375 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.910383 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.918875 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1474/2929] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:59.923071 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.933497 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fc2adf-7514-4842-b9f0-860be5f2dc62 admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.938248 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.938871 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.939186 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.939461 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] PATH_INFO: `/v3/roles` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.940506 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.942085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e54d2d4-1f0f-4a98-a732-75848698c625 admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:15:59.953021 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1456/2930] 10.222.0.97 () {58 vars in 1122 bytes} [Thu Sep 8 14:15:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 08 14:15:59.957345 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:15:59.969513 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5004a-db4f-45bb-806d-675a35631400 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:15:59.974189 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:15:59.975454 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:15:59.975728 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:15:59.976097 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] PATH_INFO: `/v3/projects/5b2c7e58a80c46b4af9b5a1022e85b43/users/d0fae7b3927e46a7bdfef441dd110d0b/roles/2061abea01db4efb90f840ed1ff9465d` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:15:59.988605 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:15:59.992790 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b2c7e58a80c46b4af9b5a1022e85b43, user_id=d0fae7b3927e46a7bdfef441dd110d0b, role_id=2061abea01db4efb90f840ed1ff9465d)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:15:59.995451 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:00.134824 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1475/2931] 10.222.0.97 () {60 vars in 1483 bytes} [Thu Sep 8 14:15:59 2022] PUT /identity/v3/projects/5b2c7e58a80c46b4af9b5a1022e85b43/users/d0fae7b3927e46a7bdfef441dd110d0b/roles/2061abea01db4efb90f840ed1ff9465d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:16:00.145276 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] REQUEST_METHOD: `POST` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:00.145558 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:00.145831 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:00.175778 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.auth.core [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] MFA Rules not processed for user `d0fae7b3927e46a7bdfef441dd110d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=88421) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:16:00.177355 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:00.257844 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1457/2932] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:16:00.270835 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:00.273177 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:00.332804 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078d201f-33bc-42f7-b485-9efd133412c4 admin admin] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:00.337018 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9f7790a1b25740fba9cb92c1fa53ba2e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:00.337618 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] REQUEST_METHOD: `GET` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:00.337896 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:00.338164 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:00.347368 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:00.348965 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84290b5f-287e-42b6-a999-5ba4acf72d27 service glance] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:00.358860 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1476/2933] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:00 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:10.253851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:10.354431 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=29,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:30.667650 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:30.733291 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe14fcb-1de9-4af4-8449-39988f268f81 None None] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:30.737885 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:30.738548 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] REQUEST_METHOD: `DELETE` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:30.738829 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:30.739104 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] PATH_INFO: `/v3/users/d0fae7b3927e46a7bdfef441dd110d0b` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:30.740851 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0fae7b3927e46a7bdfef441dd110d0b)` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:30.742759 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:30.977086 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0fae7b3927e46a7bdfef441dd110d0b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:30.977825 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0fae7b3927e46a7bdfef441dd110d0b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:30.985057 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0fae7b3927e46a7bdfef441dd110d0b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:30.997356 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0fae7b3927e46a7bdfef441dd110d0b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:30.998085 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.notifications [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0fae7b3927e46a7bdfef441dd110d0b'} {{(pid=88421) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:31.083551 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1458/2934] 10.222.0.97 () {58 vars in 1224 bytes} [Thu Sep 8 14:16:30 2022] DELETE /identity/v3/users/d0fae7b3927e46a7bdfef441dd110d0b => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:16:31.088022 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304718-0263-420b-834b-cd830e58f4dc None None] REQUEST_METHOD: `POST` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:31.088595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304718-0263-420b-834b-cd830e58f4dc None None] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:31.088595 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42304718-0263-420b-834b-cd830e58f4dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:31.107774 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.auth.core [None req-42304718-0263-420b-834b-cd830e58f4dc None None] MFA Rules not processed for user `d6d84dfa12274652b8eb04c682b6e60e`. Rule list: `[]` (Enabled: `True`). {{(pid=88422) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 08 14:16:31.108991 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-42304718-0263-420b-834b-cd830e58f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:31.169559 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1477/2935] 10.222.0.97 () {58 vars in 958 bytes} [Thu Sep 8 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 08 14:16:31.179257 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:31.180696 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:31.239903 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620d4c73-c5e2-4d6d-b850-5c2a0c24052e admin admin] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:31.243513 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f87892da36634672bac2c4c12a52f15e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:31.244122 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:31.244438 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:31.244715 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:31.254350 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:31.255801 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:31.265512 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1459/2936] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:16:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:16:31.736922 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42304718-0263-420b-834b-cd830e58f4dc None None] Authenticating user token {{(pid=88422) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:16:31.738470 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.fernet_utils [None req-42304718-0263-420b-834b-cd830e58f4dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88422) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:16:31.751184 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42304718-0263-420b-834b-cd830e58f4dc None None] Validating token access rules against request {{(pid=88422) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:16:31.756795 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6d84dfa12274652b8eb04c682b6e60e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5225debdf3f04c89923ec01541ea8b94', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88422) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:16:31.757549 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:16:31.758634 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:16:31.759338 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] PATH_INFO: `/v3/projects/5b2c7e58a80c46b4af9b5a1022e85b43` {{(pid=88422) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:16:31.760653 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b2c7e58a80c46b4af9b5a1022e85b43)` {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:16:31.762709 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] RBAC: Authorization granted {{(pid=88422) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:16:31.935624 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b2c7e58a80c46b4af9b5a1022e85b43', 'request_id': 'req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:31.942039 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b2c7e58a80c46b4af9b5a1022e85b43', 'request_id': 'req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:31.948493 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b2c7e58a80c46b4af9b5a1022e85b43', 'request_id': 'req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a'} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:32.100812 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] The token cache is being invalidate because project 5b2c7e58a80c46b4af9b5a1022e85b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=88422) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 08 14:16:32.101478 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:32.101664 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG keystone.notifications [None req-7e39c5e7-c7cc-4663-be83-7d69f3573c3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=88422) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 08 14:16:32.103239 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: [pid: 88422|app: 0|req: 1478/2937] 10.222.0.97 () {58 vars in 1233 bytes} [Thu Sep 8 14:16:31 2022] DELETE /identity/v3/projects/5b2c7e58a80c46b4af9b5a1022e85b43 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 08 14:16:41.261196 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:16:42.097489 dv-cinder-856463-2-iscsi devstack@keystone.service[88422]: DEBUG dbcounter [-] [88422] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=88422) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 08 14:20:48.744944 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] Authenticating user token {{(pid=88421) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 08 14:20:48.746354 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:20:48.885179 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804b8410-4d21-47e0-affe-cb582b9ee66a service neutron] Validating token access rules against request {{(pid=88421) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 08 14:20:48.947576 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '59c91fec50c6477b8f7b7bc45534a2d5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5a3348fedcb34424aba1f8f7828968cb', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=88421) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 08 14:20:48.948200 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] REQUEST_METHOD: `GET` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 08 14:20:48.948503 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] SCRIPT_NAME: `/identity` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 08 14:20:48.948774 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=88421) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 08 14:20:48.950795 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.fernet_utils [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=88421) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 08 14:20:48.962700 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 08 14:20:48.964288 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ca57ac-1c46-49eb-845e-a2022daaf83a service placement] RBAC: Authorization granted {{(pid=88421) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 08 14:20:48.974679 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: [pid: 88421|app: 0|req: 1460/2938] 10.222.0.97 () {60 vars in 1365 bytes} [Thu Sep 8 14:20:48 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 08 14:20:58.970121 dv-cinder-856463-2-iscsi devstack@keystone.service[88421]: DEBUG dbcounter [-] [88421] Writing DB stats keystone:SELECT=89 {{(pid=88421) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}